-- Logs begin at Wed 2022-07-27 14:37:58 UTC, end at Wed 2022-07-27 15:47:42 UTC. -- Jul 27 15:14:25.077038 dv-cinder-841315-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 15:14:25.086086 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 15:14:25 2022] *** Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: nodename: dv-cinder-841315-2-iscsi Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: machine: x86_64 Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: clock source: unix Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: pcre jit disabled Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: detected number of CPU cores: 4 Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: current working directory: / Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: detected binary path: /usr/bin/uwsgi-core Jul 27 15:14:25.088190 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: your processes number limit is 47680 Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: your memory page size is 4096 bytes Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: detected max file descriptor number: 2048 Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: lock engine: pthread robust mutexes Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: thunder lock: enabled Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 15:14:25.088822 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 15:14:25.126846 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: Python main interpreter initialized at 0x55743f19a9c0 Jul 27 15:14:25.126846 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: python threads support enabled Jul 27 15:14:25.126846 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: your server socket listen backlog is limited to 100 connections Jul 27 15:14:25.126846 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: your mercy for graceful operations on workers is 90 seconds Jul 27 15:14:25.127167 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 15:14:25.127241 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: *** Operational MODE: preforking *** Jul 27 15:14:25.127514 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: *** uWSGI is running in multiple interpreter mode *** Jul 27 15:14:25.127514 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: spawned uWSGI master process (pid: 80172) Jul 27 15:14:25.127702 dv-cinder-841315-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 15:14:25.128778 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: spawned uWSGI worker 1 (pid: 80182, cores: 1) Jul 27 15:14:25.128778 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: spawned uWSGI worker 2 (pid: 80183, cores: 1) Jul 27 15:14:25.128778 dv-cinder-841315-2-iscsi devstack@keystone.service[80172]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 15:14:30.617566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 15:14:30.617566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: from cryptography.utils import int_from_bytes Jul 27 15:14:30.618859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 15:14:30.618859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: from cryptography.utils import int_from_bytes Jul 27 15:14:30.619850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 15:14:30.619850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: from cryptography.utils import int_from_bytes Jul 27 15:14:30.620312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 15:14:30.620312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: from cryptography.utils import int_from_bytes Jul 27 15:14:30.667884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 15:14:30.668058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 15:14:30.668257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] command line args: [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 15:14:30.668490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 15:14:30.668575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 15:14:30.668683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 15:14:30.668775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ================================================================================ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 15:14:30.668879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] command line args: [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 15:14:30.668957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] admin_token = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 15:14:30.669229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_dir = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ================================================================================ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 15:14:30.669414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_source = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] admin_token = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] debug = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_dir = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.669890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_source = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] debug = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.670962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_config_append = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_dir = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.671918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_config_append = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_dir = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.672895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.673937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.674984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.675818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] publish_errors = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] publish_errors = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.676848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] standard_threads = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] standard_threads = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.677819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_journal = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_json = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_stderr = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_journal = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.678856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_syslog = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_json = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_stderr = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_syslog = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.679571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 15:14:30.679683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.679806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.679940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.680646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.680775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.680895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.681966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.682995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.683823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.684956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.685908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.686965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.687949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.688998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.689969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.690879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.691923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.692996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.693940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.694993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.695978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.696970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.697998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.698929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.699975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.700664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.701930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.702988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.703968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.external = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.704999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.external = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.password = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.705961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.password = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.token = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.token = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.706899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707842 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.707960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.708892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.709935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.710983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.711929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.712988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.713931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.714947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.715954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.716126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.716301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.716466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.716666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.716852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.717044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.717235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.717423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.717615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.717814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.718921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.719111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.719307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.719501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.719693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.719904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.720116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.720324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.720524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.720710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.720872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.721925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.722947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.723782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.724977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.725942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.726736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.727946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.728944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.729951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.730966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.731900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.732956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.733915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.734874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.735817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.736939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.737974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.738935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.739918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.740974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.741926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.742004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.742174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.742252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.742409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.742484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.743959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.744976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.745924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.746877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.747872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.748947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.749914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.750966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.751926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.752871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.753963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.754060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.754128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 15:14:30.754212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.754443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.754683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.754918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 15:14:30.755088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 15:14:30.783154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.783380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.783601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.783687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.783889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.784122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.792494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.793003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.822544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.822775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.822859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.823092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.876119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.876278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.927068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.927297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.927725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.927990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.993957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.994083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.994208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.994343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.994470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:30.994470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.017986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.018856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.030338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.030496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.043559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.043797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.043929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.044922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.045012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.045134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 15:14:31.061909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.062119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 15:14:31.062328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.062457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.062555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 15:14:31.062661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 15:14:31.062855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.062938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 15:14:31.063177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.063822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.064112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.073415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.073677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.073761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.073947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.074028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.074212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.074294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.074488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.081423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.081633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.086111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.086776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.086919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.087912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.088969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.089126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.089234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.089391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.112804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.112898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.114270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.114353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.114547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.114631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.119147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.119232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.119425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.119507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.119701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.119786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.126337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.126434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.126568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.126661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.130653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.130766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.130936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.131923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.132734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.148483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.148645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.149271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.149440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.153172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.153365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.156299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.156537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.156620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.156831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.160947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.161237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.164178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.164954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.167905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.168833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.169021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.169104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.169301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.169382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.169589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.185253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.185528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.185687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.185770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.185970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.186053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.186236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.186494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.192863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.193380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.194399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.194667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.194920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.195194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.197980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.198255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.198541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.198623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.198820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.198910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.199100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.199184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.199381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.199667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.210849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.211129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.211979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.212259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.212881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.213065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.213157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.213336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.213607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.228610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.229368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.231533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.232381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.234476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.235304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.236902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.237769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.238442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.238725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.239004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.239422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.239731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.240045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.245476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.246537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.247048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.248166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.249961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.250246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.250527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.250801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.251074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.251363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.251656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.251964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.259307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.259587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.259885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.260552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.260838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.261121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.266667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.266945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.267222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.267496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.267778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.267967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 15:14:31.268078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.268959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.269904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.270061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.270279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.270369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.270561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.270842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.271113 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.271386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.271648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 15:14:31.303836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55743f19a9c0 pid: 80182 (default app) Jul 27 15:14:31.305470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55743f19a9c0 pid: 80183 (default app) Jul 27 15:14:31.307115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f30e3-f502-438a-9398-9bd1089daeb1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:31.307402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f30e3-f502-438a-9398-9bd1089daeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:31.307677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f30e3-f502-438a-9398-9bd1089daeb1 None None] PATH_INFO: `/v3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:31.311936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Jul 27 15:14:31 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:38.979253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:38.979749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:38.979871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:38.984455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:38.989557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870120-3fc7-42f7-8581-59055005800d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:38.989945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870120-3fc7-42f7-8581-59055005800d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:38.990131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870120-3fc7-42f7-8581-59055005800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:39.059321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: INFO dbcounter [None req-da870120-3fc7-42f7-8581-59055005800d None None] Registered counter for database keystone Jul 27 15:14:39.080133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG oslo_db.sqlalchemy.engines [None req-da870120-3fc7-42f7-8581-59055005800d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80182) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 15:14:39.086916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writer thread running {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 15:14:39.262527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG passlib.handlers.bcrypt [None req-da870120-3fc7-42f7-8581-59055005800d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80182) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 15:14:39.262859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG passlib.handlers.bcrypt [None req-da870120-3fc7-42f7-8581-59055005800d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80182) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 15:14:39.316738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-da870120-3fc7-42f7-8581-59055005800d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:39.318381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-da870120-3fc7-42f7-8581-59055005800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:39.440382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 15:14:39.577298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:39.645897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: INFO dbcounter [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] Registered counter for database keystone Jul 27 15:14:39.664923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG oslo_db.sqlalchemy.engines [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80183) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 15:14:39.671055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writer thread running {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 15:14:39.789773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81732bc9-ee48-4f33-ad13-4808cbe078ea None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:39.794263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:39.794903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:39.795187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:39.795463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:39.803407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:39.854920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 15:14:39.855264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da919-8009-4b72-bcaa-62359eb178ce admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:39.864617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 15:14:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:14:39.871083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da870120-3fc7-42f7-8581-59055005800d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:39.886060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da870120-3fc7-42f7-8581-59055005800d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:39.890515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:39.891607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:39.891933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:39.892214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] PATH_INFO: `/v3/projects/c1de4e10bc0744aea22a689aefa058d1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:39.893716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] RBAC: Authorizing `identity:get_project(project_id=c1de4e10bc0744aea22a689aefa058d1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:39.943256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 15:14:39.943607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695f82e-5df3-4d76-9aa0-b5a90fb3573c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:39.948790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Jul 27 15:14:39 2022] GET /identity/v3/projects/c1de4e10bc0744aea22a689aefa058d1 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:41.904119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e9e0-57d7-4a77-bb6a-45533e370bf7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:41.904659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e9e0-57d7-4a77-bb6a-45533e370bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:41.904659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e9e0-57d7-4a77-bb6a-45533e370bf7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:41.908715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:14:41.914057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:41.914448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:41.914595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:41.934125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:41.935623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:41.943327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 15:14:41.965732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05e9e0-57d7-4a77-bb6a-45533e370bf7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:41.977741 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05e9e0-57d7-4a77-bb6a-45533e370bf7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:41.981836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:41.982436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:41.982718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:41.982991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:41.990531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:41.992304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e93bc3-66ec-4e4c-a6a7-77bd7b39e9e8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:42.000858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 15:14:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:14:42.006007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:42.015536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b5d3-380c-4371-a6ad-6f8e637f45ee None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:42.019925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:42.020555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:42.020840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:42.021120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:42.022484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:42.024959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c99d1-2bae-4e98-91ea-4ed321ac1476 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:42.026493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Jul 27 15:14:42 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:44.078805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c4f0-42f9-4bff-9630-8ee1aa31780d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.079321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c4f0-42f9-4bff-9630-8ee1aa31780d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.079413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c4f0-42f9-4bff-9630-8ee1aa31780d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.080733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:44.084403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8379-abf4-4fe0-8e6f-94ec4be900eb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.084818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8379-abf4-4fe0-8e6f-94ec4be900eb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.084906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8379-abf4-4fe0-8e6f-94ec4be900eb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.085957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.086258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:44.086342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.086615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.091038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.091329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.091613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.110631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:44.111842 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:44.128993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG passlib.handlers.bcrypt [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80183) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 15:14:44.129408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG passlib.handlers.bcrypt [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80183) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 15:14:44.174626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:44.176237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:44.190586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 7/12] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:14:44.212450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.235903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.278558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 6/13] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:44.301134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.316147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028f0fe-d75b-47f2-8677-f1de6756ba03 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.350757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.351371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.351652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.351956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.401685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224c987-6222-4e31-ba7b-cfe4e6aeebcd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.403966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.417850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.419363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.440326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.441416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.441702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.441978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.442519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.454931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:44.463525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:44.464510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:14:44.465505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:14:44.470596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.519210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 8/15] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:14:44.524455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.567849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58ba6-880a-44c4-9a2e-acba56b4d818 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.601264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.601879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.602163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.602436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.603149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:44.603451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.604918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.616798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 8/16] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:44.620861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787c636-9ac8-47b8-8d35-1d60167141a5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.622737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.643529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.649983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.656732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.657379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.657666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.657943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.668464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.669933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:14:44.688083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:44.689117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:44.690105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:14:44.690105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:14:44.690105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:14:44.690105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:14:44.690105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:14:44.696548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.723921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42330daa-dfec-4fc3-b4e5-3c86baf7e0cf None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.758392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.759034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.759311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.759592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.761145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.762630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.764276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:44.769636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.868481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:44.875409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a0c14-edeb-4a20-b6be-6d8cecef3e38 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.902789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca068-d120-4732-9a33-29c84a5627e9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:44.909457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.910098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.910383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.910660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.911818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.913305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.915405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:44.921125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:44.949249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:44.949887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:44.950171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:44.950448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:44.951415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:44.951723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:44.953182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b716d0-e3cf-40be-9215-bc30e5b32193 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:44.970479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:45.034890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffcb83-901d-4d66-b8ee-a7e6c8186531 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:45.070940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:45.071568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:45.071875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:45.072152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:45.072948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.domain.id=default, user.id=feddce10ac514a48a58f8b731d31a20b) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:45.073257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:45.074732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0fa80-6911-4755-898e-2c6054780d19 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:45.084272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 15:14:44 2022] GET /identity/v3/role_assignments?user.id=feddce10ac514a48a58f8b731d31a20b&scope.domain.id=default&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:47.221094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d8a1e-c451-4034-9f2b-004f319fc14b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.221929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d8a1e-c451-4034-9f2b-004f319fc14b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.222356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d8a1e-c451-4034-9f2b-004f319fc14b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.223880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:47.229214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.229987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.230436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.259608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:47.260581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947786-d9b9-4a9e-90b2-a7adda475541 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.260874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:47.261247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947786-d9b9-4a9e-90b2-a7adda475541 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.261690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947786-d9b9-4a9e-90b2-a7adda475541 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.263428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:14:47.276503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.277339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.277822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.292343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:47.315855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:47.317062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:47.362940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:47.377781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:47.385769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:47.448631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:14:47.486419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:47.636325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc82f-a482-4133-a602-8fe6b6f4bcd4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:47.662385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0835898-d1b2-4d09-bb9f-727bdb0f5b45 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:47.698808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:47.699694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.700156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.700856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.711512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:47.713286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:14:47.734557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:47.735574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:47.736665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:14:47.736665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:14:47.736665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:14:47.736665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:14:47.738277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 14/26] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:14:47.744690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:47.760566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:47.761232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.761703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.762129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.772178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:47.783197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:47.788321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:47.789701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:14:47.790466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 13/27] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:14:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:47.796837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:47.849316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188922-8681-4eea-9a09-7efa5db6e46d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:47.860209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:47.889402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:47.890259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.890682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.891263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.892177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:47.892641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:47.894155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a762f8-d24c-40fd-9f3c-d02ed607e09f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:47.905973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 15/28] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:47.907372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1462-9422-488f-918e-e7f9e1643ca0 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.907794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1462-9422-488f-918e-e7f9e1643ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.908253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1462-9422-488f-918e-e7f9e1643ca0 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.909592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 16/29] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:47.912023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c1462-9422-488f-918e-e7f9e1643ca0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:47.941465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495d1d-ef32-426a-8511-f9b6723c61a7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:47.980536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:47.981165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:47.981449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:47.981725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:47.982447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:47.982753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:47.989398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74b940-abfd-4b93-a3cd-896b6f98c699 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.025750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 14/30] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:48.027186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d436a-edf6-44db-ba6a-ba7d015f3bdd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.027478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d436a-edf6-44db-ba6a-ba7d015f3bdd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.027752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d436a-edf6-44db-ba6a-ba7d015f3bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.056250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:48.082310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c1462-9422-488f-918e-e7f9e1643ca0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:48.121472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:48.122353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.122805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.123248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.124125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:48.125777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.210568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-112d436a-edf6-44db-ba6a-ba7d015f3bdd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:48.212689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-112d436a-edf6-44db-ba6a-ba7d015f3bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:48.282568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 17/31] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:14:47 2022] POST /identity/v3/projects => generated 301 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:48.284341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:48.293240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 15/32] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:48.294616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582fd-1a42-4525-9b99-dece635f9820 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.294917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582fd-1a42-4525-9b99-dece635f9820 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.295199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582fd-1a42-4525-9b99-dece635f9820 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.297112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 16/33] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:48.302543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.302827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.303268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.322921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:48.324390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:48.326944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:48.403088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 17/34] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:48.403267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2ea6-4bda-4de2-9de4-2e2bb125a079 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:48.404394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:48.434644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:48.443092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:48.444248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.444854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.445285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.447788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:48.449852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.452206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 18/35] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:48 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:48.453381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:48.552414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419499a-8379-4e93-a233-da24dbfd0905 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:48.561567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3063ad-bc6a-4950-9129-19c0c5c7a335 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:48.594473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:48.595438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.595995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.596441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.597302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:48.598821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569abfb2-8880-40ff-9ad3-26c1c8888f3c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.601309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:48.602208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.602707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.603171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.604113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:48.606474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.611352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:48.630831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 19/36] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 15:14:48 2022] POST /identity/v3/roles => generated 225 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:48.636041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:48.729016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 18/37] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 15:14:48 2022] POST /identity/v3/roles => generated 219 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:48.735868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:48.751018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e543-1e8a-40cb-b26a-4e76742f52cb None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.751666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e543-1e8a-40cb-b26a-4e76742f52cb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.752214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e543-1e8a-40cb-b26a-4e76742f52cb None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.753916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 19/38] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:14:48.765389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.766005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.766475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.804523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8205e-3de4-4fc2-bf99-3abae335ca49 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:48.806280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:48.808002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:48.850508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:48.851790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:48.852280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:48.852706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:48.854814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:48.856632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:48.859702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 20/39] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:48.865242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:48.978275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 20/40] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:14:49.001785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:49.025603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eb849-c0a8-4a91-b02d-36103dff44dd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:49.059686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:49.072318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:49.073006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.073292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.073579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] PATH_INFO: `/v3/domains/default/users/feddce10ac514a48a58f8b731d31a20b/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.076654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=feddce10ac514a48a58f8b731d31a20b, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:49.078120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308677cd-4f6f-48b9-8e46-607b405a95f2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:49.092617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 21/41] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Jul 27 15:14:48 2022] PUT /identity/v3/domains/default/users/feddce10ac514a48a58f8b731d31a20b/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:14:49.093947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe94f2-a6d1-432f-8614-f75f42b70c34 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.094270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe94f2-a6d1-432f-8614-f75f42b70c34 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.094520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe94f2-a6d1-432f-8614-f75f42b70c34 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.100012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 22/42] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:49.101610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.101901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.102239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.121878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:49.123474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:49.131264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace5f26-e88f-415f-a3c3-be94a8eee815 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:49.163678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:49.174248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:49.174902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.175189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.175478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.176272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:49.177988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:49.197788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 21/43] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 15:14:49 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:49.203902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 23/44] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:14:49.224260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:49.346957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984b5a2-ccc0-463c-9b38-f19d415e134e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:49.385722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:49.386653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.386952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.387326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.388540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:49.390007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245941a6-55cb-4df4-ae80-66e626db165f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:49.391789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 22/45] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:49.397263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:49.501235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8435008-dfb4-4b33-9845-2637b4950f85 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:49.514923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:49.536972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:49.537602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:49.537893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:49.538174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:49.538875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:49.540908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f394-4793-42c4-b479-41d5e0d93b7e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:49.566915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:14:49 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:51.127667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b270-842d-4811-8654-a8e3749b4576 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.128237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b270-842d-4811-8654-a8e3749b4576 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.128237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b270-842d-4811-8654-a8e3749b4576 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.129603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 23/47] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:51.134796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.135191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.135485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.153762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:51.154951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:51.220337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 25/48] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:51.243150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b270-842d-4811-8654-a8e3749b4576 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.300760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:51.345624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b270-842d-4811-8654-a8e3749b4576 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.378348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.378991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.379280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.379560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.390251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:51.395328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:51.396205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:14:51.397199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 24/49] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:51.402824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.494902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:51.501533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c0216-7959-4d6f-8710-e5d4e1b10df8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.534970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.535585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.535892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.536171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.536885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:51.537188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.538565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980926a-b747-47ce-b24f-40ebd9901c33 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.551052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 26/50] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:51.556610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.573885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa3ad-56aa-4f8b-8c8d-97659da36d77 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.574179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa3ad-56aa-4f8b-8c8d-97659da36d77 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.574456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa3ad-56aa-4f8b-8c8d-97659da36d77 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.576278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 27/51] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:14:51.581826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.582112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.582390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.600322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:51.601567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:51.658270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859d9ea-75c9-4c4d-a038-c3fa171c8938 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.668505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 28/52] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:14:51.673828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:51.691182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.691942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.692679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.692967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.693249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.694605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.696729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.698462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 25/53] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:51.704219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.780542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:51.793193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c605b8-9f77-41e3-b2ab-8cafbf15c09b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.805539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c58c-e3d7-406c-b512-4e47ff6b66ac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.827287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.827945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.828235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.828506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.829674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.831116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.832714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 29/54] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:51.838244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.838722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.839361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.839648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.839960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.841151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.842595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.844218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 26/55] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:14:51.849623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:51.901605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:51.936594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57309b-61cd-41e0-8f1e-0c3597cd4313 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.952472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382dab-d9b9-42c2-9e86-2eb5a0484a84 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:51.970457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.971079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.971366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.971647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.972370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.973836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48105f1-a881-43a6-bdf9-adafd8f10377 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.985703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:51.986385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:51.986676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:51.986958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:51.987743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.domain.id=default, user.id=feddce10ac514a48a58f8b731d31a20b) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:51.988084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:51.989498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83307e2c-7919-43b6-952f-fdc146e366b0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:51.996616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 27/56] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 15:14:51 2022] GET /identity/v3/role_assignments?user.id=feddce10ac514a48a58f8b731d31a20b&scope.domain.id=default&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:51.999787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 30/57] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 15:14:51 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:14:54.011578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54531-b99c-40bd-a012-2376317a33ec None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:54.012067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54531-b99c-40bd-a012-2376317a33ec None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:54.012247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54531-b99c-40bd-a012-2376317a33ec None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:54.013610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 28/58] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:54.018773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:54.019135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:54.019342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:54.037563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:54.038744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:54.040699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:54.103925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 31/59] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:54.127314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc54531-b99c-40bd-a012-2376317a33ec None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:54.226008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc54531-b99c-40bd-a012-2376317a33ec None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:54.242414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:54.261207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:54.261945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:54.262746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:54.263443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:54.264623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:54.266896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caad32-1666-4b54-9516-75107cb8c1f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:54.269309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 29/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:54.275013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:54.370746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efd74b-f8ed-495c-9a63-053597acf16b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:54.403257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:54.403910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:54.404192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:54.404464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:54.405144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:54.406592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b185d-1d58-45e4-aedf-15f407c3dca6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:54.420720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:54.433852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 32/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:14:54 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:14:57.352218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f301ce-332a-4c89-a82d-467bb6469065 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.353380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f301ce-332a-4c89-a82d-467bb6469065 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.353862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f301ce-332a-4c89-a82d-467bb6469065 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.355422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 30/62] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:57.360805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.361597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.362080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.388849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:57.390366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:57.480239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 33/63] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:57.517368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f301ce-332a-4c89-a82d-467bb6469065 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:57.675329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f301ce-332a-4c89-a82d-467bb6469065 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:57.720677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:57.721674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.722154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.722623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.743228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b5bccd43-992c-467b-8f4d-5713b2f6d4f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:57.748978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:57.749817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:14:57.750557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:14:57.752838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 31/64] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:57.759016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:57.818884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98d41-dc71-4c31-95e6-4c3e4cd75177 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.819650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98d41-dc71-4c31-95e6-4c3e4cd75177 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.820570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98d41-dc71-4c31-95e6-4c3e4cd75177 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.822472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 32/65] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:57.831170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd920-18ce-4044-aa08-00c57d640a08 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.831170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd920-18ce-4044-aa08-00c57d640a08 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.831170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd920-18ce-4044-aa08-00c57d640a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.850984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bd0cd920-18ce-4044-aa08-00c57d640a08 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:57.853033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd0cd920-18ce-4044-aa08-00c57d640a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:57.900176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:57.907088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02939fd2-2435-4859-a6eb-1f9a9875fab2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:57.931992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 33/66] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:57.933492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a701a7-67fb-43b9-aa86-ec8c77f3c6ca None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.933939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a701a7-67fb-43b9-aa86-ec8c77f3c6ca None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.934350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a701a7-67fb-43b9-aa86-ec8c77f3c6ca None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.935808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 34/67] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:57.942064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.942518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.942964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.944204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:57.957519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:57.958150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.958440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.958721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.959453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:57.959765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:57.961719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b1f78-8957-4ac2-821b-a85a04c3b347 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:57.971046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 34/68] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:14:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:57.971186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:57.972328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b071-06a6-475d-bf18-4dd0a59ed0fa None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:57.972770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b071-06a6-475d-bf18-4dd0a59ed0fa None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:57.973193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5b071-06a6-475d-bf18-4dd0a59ed0fa None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:57.973303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:57.974674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 35/69] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:57.980085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5b071-06a6-475d-bf18-4dd0a59ed0fa None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.046916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 35/70] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:58.048175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.093506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5b071-06a6-475d-bf18-4dd0a59ed0fa None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.128050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.128683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.128964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.129239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.141357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-eeb8ff96-44f1-4559-ac1e-f086f07ec3fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:58.147760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:58.149151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 36/71] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:14:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:58.152453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.153147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.153147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.168344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322c248-6507-4d81-9a42-c87ba331c3be None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.169998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:58.171190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:58.171858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.207544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.208142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.208425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.208705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.215676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.224336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.225780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.227491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 36/72] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.228275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.240047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 37/73] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:58.245076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.327582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394aa28-7a88-4eb5-9cff-9396e3dbae46 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.342687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239a692-86ed-4d1b-a9aa-c72c795c2bb9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.360683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.361331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.361619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.361902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.371995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:58.375408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.376060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.376342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.376618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-764b99a7-6fbe-4620-be38-e824cc3421bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:58.377023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:58.377980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:14:58.378728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:14:58.379300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:58.379300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.379415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 37/74] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:58.379476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.379537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef506e6-b9dd-4fe7-bd3e-5f027bb597d6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.379618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef506e6-b9dd-4fe7-bd3e-5f027bb597d6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.380129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef506e6-b9dd-4fe7-bd3e-5f027bb597d6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.381127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 38/75] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:14:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:14:58.381833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef506e6-b9dd-4fe7-bd3e-5f027bb597d6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.391377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 38/76] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.392400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.409238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.433615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.486098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef506e6-b9dd-4fe7-bd3e-5f027bb597d6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.493757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b71ef-8141-4e41-8f9d-3e9111384932 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.519996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.520611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.520899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.521182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.527058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.527687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.528001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.528275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.538884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:14:58.544250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:14:58.545086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:14:58.545865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 39/77] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:14:58.546574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.592522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.593984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.616409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.640107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.644290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243c49-47c1-49a6-b213-74e79a2d086f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.678501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.679119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.679980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.680250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.680973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:58.681283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.682699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11a89a-7ed9-4d78-ac6f-f152ee4a2fd1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.690746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 40/78] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.691920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.692218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.692494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.692852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 39/79] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:14:58.693709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.710997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:14:58.712211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:14:58.776418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 41/80] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:14:58.777161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.799512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311aa37-f4ff-4852-858f-7f9332d3bee9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.823174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.835385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.836563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.836851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.837124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.838280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.839709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.841311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 40/81] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.842067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.878114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:58.890016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea041a5-669f-4c31-89f1-23a2b1c81423 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.924161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.924785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.925066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.925349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.926264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:58.926385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.927707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.935946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 42/82] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.936772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:58.947321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeee38-f995-4226-8197-800f9c8ae1c7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:58.985744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:58.986392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:58.986673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:58.986951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:58.988138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:58.989567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:58.991211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 41/83] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:58.992019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.048212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.048707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddce8-573b-4e93-b748-c599cbd79847 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.082630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.083267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.083554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.083856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.085088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.087096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.088874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 43/84] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:59.089485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.089573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ce7c-2349-499c-8c21-b03f30f5a999 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.110915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.125932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.126573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.126860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.127139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.128325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.129756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.131331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 42/85] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:14:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:59.132124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.191205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2518a8-ce8a-416d-b64e-931bb9a95d8f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.226583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.227216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.227502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.227785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.228826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb9cee-fa83-4010-850d-46113b311027 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.253648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.264014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.264658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.264946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.265227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.266413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.267872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.269496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 43/86] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:59.270262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.297528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.299044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.318830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.366431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0902-67f3-4037-b64b-cd4dc7482d39 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.398041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 44/87] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:14:59.398879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.401014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.401579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.401874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.402741 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.402944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:14:59.403258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.404726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.410170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 44/88] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:59.410929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.476837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.514577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0aed5-09bb-4e88-b0b7-48cc9d6b416c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.529801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab6201-723a-40f3-8f9b-23506bfd6d8c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.536550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.555081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.555617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.555926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.556203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.573708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.574646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.575124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.575671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.576477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.577930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.607241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 45/89] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:14:59 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:14:59.608692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.637831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.639350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.758703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 45/90] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:14:59.759556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:14:59.770612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522ecc-6901-4b62-88b1-5c554357531c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.783445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.787792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:14:59.828655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.829373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.829661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.829941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.893964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db30754-4f81-4780-bce9-05176c595839 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:14:59.919538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.921382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.932344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:14:59.932998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:14:59.933274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:14:59.933556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:14:59.935245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:14:59.937733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:14:59.939640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 46/91] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:14:59.940416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:00.045637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:00.059607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6feece-407d-4c56-87e7-fa11b07a370a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:00.060537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 46/92] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:00.061812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:00.081922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:00.103532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:00.104223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:00.104506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:00.104783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:00.106064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:00.107651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:00.110612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 47/93] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:14:59 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:00.112245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:00.207740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6852b3-d666-4e88-b0e4-e57c91626165 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:00.240815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b91d-c115-4ed7-af1e-adceec3014aa None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:00.277773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:00.278424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:00.278707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:00.278985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] PATH_INFO: `/v3/projects/d7e512911092497dbabb751ef98f110e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:00.280193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] RBAC: Authorizing `identity:get_project(project_id=d7e512911092497dbabb751ef98f110e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:00.281972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:00.283983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 48/94] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:00 2022] GET /identity/v3/projects/d7e512911092497dbabb751ef98f110e => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:00.284879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:00.292824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:00.293469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:00.293757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:00.294038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:00.294828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:00.295138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:00.296592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:00.312049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:00.313846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 47/95] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:00 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:00.314615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:00.494867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30e80-7511-459d-adfd-d2b0791ef0ad None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:00.521094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:00.559143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b9b3f-79df-47a8-8b67-7104db9e51fc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:00.565461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:00.566112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:00.566400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:00.566683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:00.567495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=feddce10ac514a48a58f8b731d31a20b) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:00.567812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:00.569454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa39908-4edd-4bd1-98a9-1eec9dd4c9ec None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:00.582940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 49/96] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:00 2022] GET /identity/v3/role_assignments?user.id=feddce10ac514a48a58f8b731d31a20b&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:00.602361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:00.603352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:00.603906 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:00.604404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:00.605421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=d7e512911092497dbabb751ef98f110e, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:00.605933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:00.607576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336e3ee-9859-40d5-b20d-110e0d365967 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:00.614156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 48/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:00 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=d7e512911092497dbabb751ef98f110e&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:01.874100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf365-c901-4049-bc1e-b7e2ade39401 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:01.874622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf365-c901-4049-bc1e-b7e2ade39401 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:01.874704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf365-c901-4049-bc1e-b7e2ade39401 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:01.876045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 50/98] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:01.881242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:01.881648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:01.881812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:01.912781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:01.914530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:02.020356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 49/99] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:02.062846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acf365-c901-4049-bc1e-b7e2ade39401 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:02.170400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:02.190668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acf365-c901-4049-bc1e-b7e2ade39401 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:02.235318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:02.235993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.236280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.236560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.252748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:02.258166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:02.259029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:02.259819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 51/100] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:02.268954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:02.372325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:02.419497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba013ab-a258-40e5-a3ef-74228e09e878 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:02.469176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:02.469731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.470016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.470290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.471016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:02.471324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:02.472745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3364-15cd-4902-a592-1c37e38cac16 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:02.482238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 50/101] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:02.487802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:02.622698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301db8-2285-482d-9f93-aadc435ef441 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:02.666081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:02.667082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.667570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.668067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.669445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:02.671230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:02.673128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 52/102] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:02 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:02.675852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4749-a480-4da2-8eb5-e1b87fb76422 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.675852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4749-a480-4da2-8eb5-e1b87fb76422 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.676151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e4749-a480-4da2-8eb5-e1b87fb76422 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.677524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 51/103] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:02.678832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:02.682953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.683249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.683530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.704793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:02.730737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:02.732009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:02.867175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea3c2-0cd3-4421-8361-bf64505be396 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:02.874513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 52/104] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:02.911864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:02.925115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:02.925781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:02.926074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:02.926362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:02.932574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:02.934079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:02.935799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 53/105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:02 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:02.942140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.041950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.075785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.106032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa164f-4524-4e74-812a-3c1c1d3b6808 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.114738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69c83-0144-4227-b048-4e54ae2f1085 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.150388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.151071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.151357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.151642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/users/fc52d58399b846da8fd7d4217e87ec87/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.153668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, user_id=fc52d58399b846da8fd7d4217e87ec87, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.155151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42f2ac-f5d0-4cda-ae02-d3d342c396d6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:03.159626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.160449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.160767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.161024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.171995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 54/106] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:02 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/users/fc52d58399b846da8fd7d4217e87ec87/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:15:03.172894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:03.174046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a44a-adc8-4dfe-bbf1-b89d2cea0729 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.174590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a44a-adc8-4dfe-bbf1-b89d2cea0729 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.175047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a44a-adc8-4dfe-bbf1-b89d2cea0729 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.176693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 55/107] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:03.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:03.180205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:03.181370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:03.187886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 53/108] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:03.189757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.190048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.190333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.195786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.229173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:03.230409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:03.349244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b2493-c22e-4e29-95aa-6d0e80a61048 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.377941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.392446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.393697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.393989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.394267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.395170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:03.395580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.398441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90a137-4e36-4900-b4e8-2c6aa82548e1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:03.409396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 54/109] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:03.410403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 56/110] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:03.411170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6172-79c2-480e-983e-77b3b9d756b3 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.411170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6172-79c2-480e-983e-77b3b9d756b3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.411354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6172-79c2-480e-983e-77b3b9d756b3 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.412892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 55/111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:03.416497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.420428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.420720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.421006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.446658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:03.448042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:03.508234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.522743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 56/112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:03.523881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.550867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a125e2e-2ea1-47aa-94e1-c7ce37dcdb2a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.596714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.597868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.598670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.599405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] PATH_INFO: `/v3/users/42189fe97f024808be24b90468082e48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.618893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] RBAC: Authorizing `identity:get_user(user_id=42189fe97f024808be24b90468082e48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.621169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:03.623945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 57/113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:03 2022] GET /identity/v3/users/42189fe97f024808be24b90468082e48 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:03.625008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.676685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ee9a3-46aa-49e2-86de-b303e9d9b9dd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.696022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.716283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.717084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.717476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.717856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.719606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.720753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:03.722654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 57/114] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:03.723509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.761913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43d79b-6b37-4187-a41b-c4e050c34784 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.802557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.810541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.811434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.812184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.812612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.827232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e7d5cf45-cec6-4711-b039-2fe11ec7d77b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:03.833973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:03.834833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:03.836197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:03.836769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74565b5-a4b2-44bf-a0d7-a7eb8e2b9cad None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.836838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 58/115] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:03.836901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e85b2c-bff3-41ef-bc74-293bc63451a2 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.837171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e85b2c-bff3-41ef-bc74-293bc63451a2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.837450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e85b2c-bff3-41ef-bc74-293bc63451a2 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.838695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 59/116] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:03.839385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e85b2c-bff3-41ef-bc74-293bc63451a2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:03.873276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.874225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.874746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.875199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.932267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:03.951052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.955886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e85b2c-bff3-41ef-bc74-293bc63451a2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:03.956364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:03.994546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:03.995619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:03.995940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:03.996223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:03.997399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:03.999525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce903b-e629-4e36-831f-873708b472ea None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.043857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 60/117] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:15:03 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:04.043857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebb03-5925-4e59-873c-7f4087fbb7fc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.043857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebb03-5925-4e59-873c-7f4087fbb7fc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.043857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebb03-5925-4e59-873c-7f4087fbb7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.060545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b7aebb03-5925-4e59-873c-7f4087fbb7fc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:04.062402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b7aebb03-5925-4e59-873c-7f4087fbb7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:04.068007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 58/118] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:04.068952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.125655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.165413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 61/119] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:04.166654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0ae5-f429-4c0d-8a78-dbb8885cd07d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.167016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0ae5-f429-4c0d-8a78-dbb8885cd07d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.167400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0ae5-f429-4c0d-8a78-dbb8885cd07d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.168955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 62/120] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:04.169617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a0ae5-f429-4c0d-8a78-dbb8885cd07d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.207261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.232500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9012ccb-1cc1-426f-9a60-1b09b04dcb2b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.271570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.272232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.272526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.273372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.274105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:04.274418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:04.275798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dbbda-655d-402b-8073-34ee9f63c351 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.285187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 59/121] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:04.286567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.286859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.287240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.293129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a0ae5-f429-4c0d-8a78-dbb8885cd07d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.309533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:04.310898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:04.347180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.348120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.348547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.348962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.350151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:04.351975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.354744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 63/122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:04.355310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.387084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 60/123] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:04.388062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.416314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.487260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.492364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089842-6c96-4820-a3a0-69ba230e5aff None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.517843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e272f4-b8dd-4631-86ef-9f347e8b0bfc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.541071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.542001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.542430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.542925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.560001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:04.562920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.563676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.564102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.564538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.565896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:04.566286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:04.567238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:04.568034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:04.568638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.570448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 61/124] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:04.570839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 64/125] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:04.571926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.576276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.703140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d98c4-4599-4252-bb4a-a097e4e8d574 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.704596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.705041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafe83b-5a22-4ec1-a23e-161b548e9086 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.743452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.744379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.744841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.745291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.746226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=42189fe97f024808be24b90468082e48) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:04.746698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:04.747647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.748309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.748592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.748868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.749146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.755195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 62/126] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/role_assignments?user.id=42189fe97f024808be24b90468082e48&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:04.756365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.761946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:04.768032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:04.768977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:04.770015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 65/127] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:04.770639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:04.776801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:04.901185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2d6d6-c0f6-41f8-8056-6f03d94f7917 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.965151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67439b3a-7824-4e6f-ba4c-c9343206b636 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:04.966188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:04.967329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:04.967803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:04.968293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:04.976837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:04.978690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:04.981009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 66/128] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:04.981996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.011709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.012383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.012674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.012958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.013820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:05.014101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.015509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.025680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 63/129] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:05.026456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.057040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.107392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.156751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53b571-5b45-400d-94a8-889a7089b35d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.158995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1c5c4-a2e9-4a5e-a3e9-6ff49dcf9e2e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.227258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.227873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.228168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.228454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/users/feddce10ac514a48a58f8b731d31a20b/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.231150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, user_id=feddce10ac514a48a58f8b731d31a20b, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.232628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.247402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.248104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.248393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.248661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.249459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:05.249775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.251182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.260227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 64/130] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:05 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/users/feddce10ac514a48a58f8b731d31a20b/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:05.261250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.274293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 67/131] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:05.278651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.463539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.478110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c227-1d69-4997-afa5-bfd2b0c9f58c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.485248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710371-9798-488a-ab33-119bcb86d476 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.531147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.531330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.532251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.532251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.532497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.534079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.535334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.537379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 68/132] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:05 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:05.544937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.545908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.546858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.548200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.548691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.552079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.554348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef92e4-61fc-4d63-b338-b619d9b971b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.556598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 65/133] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:05 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:05.558979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52970c04-1480-4c2e-9270-4d34f2a5e093 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.559273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52970c04-1480-4c2e-9270-4d34f2a5e093 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.559556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52970c04-1480-4c2e-9270-4d34f2a5e093 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.564066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 66/134] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:05.564066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52970c04-1480-4c2e-9270-4d34f2a5e093 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.665671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42a26-3649-4d14-94c2-0bb44441fb94 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.671721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52970c04-1480-4c2e-9270-4d34f2a5e093 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.702064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.702934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.703422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.703899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] PATH_INFO: `/v3/projects/d7e512911092497dbabb751ef98f110e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.705344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e512911092497dbabb751ef98f110e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.707012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf592b2-ab4a-4107-a15d-2c497d3b0607 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.708940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 69/135] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:05 2022] GET /identity/v3/projects/d7e512911092497dbabb751ef98f110e => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:05.710404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.710581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.711067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.711168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.711458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.711740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.712065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.712936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.714520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.719854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 67/136] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:05 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:05.719854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.736479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:05.737946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:05.762418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.791994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:05.810430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 70/137] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:05.811606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.830742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b9e9-87a6-4167-b8df-3ef40deae0e4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.866053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.867411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.867702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.868005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] PATH_INFO: `/v3/projects/d7e512911092497dbabb751ef98f110e/users/fc52d58399b846da8fd7d4217e87ec87/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.869978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e512911092497dbabb751ef98f110e, user_id=fc52d58399b846da8fd7d4217e87ec87, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.871435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:05.885726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 68/138] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:05 2022] PUT /identity/v3/projects/d7e512911092497dbabb751ef98f110e/users/fc52d58399b846da8fd7d4217e87ec87/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:05.886510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:05.944337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aca148-9608-453d-91fe-8d95ca0b573d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:05.983245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:05.984749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:05.985264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:05.985744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/users/fc52d58399b846da8fd7d4217e87ec87/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:05.988428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, user_id=fc52d58399b846da8fd7d4217e87ec87, role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:05.990224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:06.003859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4e03-5c97-45ef-ac5a-2fb2c8b612f5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:06.009279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 71/139] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:05 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/users/fc52d58399b846da8fd7d4217e87ec87/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:06.041647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:06.053092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669ac7-c867-42b9-840e-845727426309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:06.053767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669ac7-c867-42b9-840e-845727426309 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.054708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669ac7-c867-42b9-840e-845727426309 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.055067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669ac7-c867-42b9-840e-845727426309 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.073443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65669ac7-c867-42b9-840e-845727426309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-65669ac7-c867-42b9-840e-845727426309 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:06.079302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:06.080212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:06.081027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 69/140] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:06.087380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:06.111163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:06.222333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4e4e5-da82-41cf-b33f-ca4f563a5313 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:06.264707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:06.265267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.265562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.265839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.266570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:06.266875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:06.268298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:06.277769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 72/141] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:06.283484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669ac7-c867-42b9-840e-845727426309 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:06.391205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669ac7-c867-42b9-840e-845727426309 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:06.430815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:06.431468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.431753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.432074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.433264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:06.434832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b81b24-5128-4bd6-9b2e-8a7d166a2fd3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:06.436932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 70/142] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:06 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:06.442348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:06.550283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:06.576948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e860d-1987-457d-bf83-4611d0d899de None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:06.615821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:06.616492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.616772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.617104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.709428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:06.711512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:06.729621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69ae7-e176-4ce0-8feb-8876c6927c53 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.729900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69ae7-e176-4ce0-8feb-8876c6927c53 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.730171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69ae7-e176-4ce0-8feb-8876c6927c53 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.731512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 71/143] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:06.736539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:06.736825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:06.737103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:06.761874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:06.763070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:06.862891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:06.878478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:06.899488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 73/144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:06.907362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:06.917931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 72/145] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:06.952349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:07.096863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372e9f-60ea-41fc-897e-f66836f55b00 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:07.138965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:07.140337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.140840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.141309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.142710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:07.144390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:07.146949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 74/146] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:06 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:07.155495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:07.170955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16389da-60c9-4ea2-acb2-dd9bdf79f347 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:07.215203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:07.234835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:07.235488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.235771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.236079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.237950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:07.242900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:07.244529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 73/147] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:07.251493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:07.321949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3307f1-48ac-4f54-b077-8608fcef4663 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:07.360235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:07.372954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:07.373996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.374381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.374754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.375789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:07.376379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:07.377912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60957-e6f2-4911-bc60-80811a7bd46c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:07.385606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 75/148] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:07 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:07.456336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027178-c218-49e5-bd80-6ac1d5e2a7ec None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:07.499134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:07.500061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.500499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.500933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.501785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:07.503399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929088d0-21a4-4692-afa9-5545dc4da715 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:07.537824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 74/149] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:07 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:07.841213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad6fb-c8aa-4134-b522-23373f69c3b5 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.841407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad6fb-c8aa-4134-b522-23373f69c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.841674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad6fb-c8aa-4134-b522-23373f69c3b5 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.843223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 76/150] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:07.849022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:07.849595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:07.850081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:07.869751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:07.871276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:07.945676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 75/151] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:07.969608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cad6fb-c8aa-4134-b522-23373f69c3b5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.076183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cad6fb-c8aa-4134-b522-23373f69c3b5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:08.097109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:08.111329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:08.112081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.112440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.112845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.123846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:08.129320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:08.130229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:08.131010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 77/152] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:08.138461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.179018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:08.243219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c117c-98c0-4d76-9a94-6d78ff0e6b13 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:08.280991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:08.281620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.281904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.282178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.282900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:08.283206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:08.284622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:08.295899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 76/153] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:08.301499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.410281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cb0dc-ccc9-4a9d-a1c8-5806b3c02a0d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:08.445765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:08.446412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.446692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.446969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] PATH_INFO: `/v3/users/42189fe97f024808be24b90468082e48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.448198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] RBAC: Authorizing `identity:get_user(user_id=42189fe97f024808be24b90468082e48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:08.449675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4474c86-3d1d-484b-afa0-002bc79312c9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:08.451569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 78/154] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:08 2022] GET /identity/v3/users/42189fe97f024808be24b90468082e48 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:08.457175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.566062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:08.571648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c9d2-9081-404c-bf21-e728ed68ba4a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:08.611955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a677-22e2-4e5d-a9e0-54bdca32af43 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.612255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a677-22e2-4e5d-a9e0-54bdca32af43 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.612347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:08.612542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a677-22e2-4e5d-a9e0-54bdca32af43 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.612935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.613221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.613501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.613866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 79/155] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:08.617331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:08.619105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.619223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:08.619392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.619656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.620922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 77/156] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:08 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:08.626883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.645668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:08.646874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:08.712551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:08.773527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 80/157] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:08.787489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf90-768c-4f8f-a8bd-2f06341519b2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:08.809421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:08.845547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:08.846236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.846501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.846784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/users/42189fe97f024808be24b90468082e48/roles/2aa3a46a15e347ceaebef17caa92ba7c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.850302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, user_id=42189fe97f024808be24b90468082e48, role_id=2aa3a46a15e347ceaebef17caa92ba7c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:08.853349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519a408-e1b9-425d-89a3-9e7ede62f889 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:08.875514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 78/158] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:08 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/users/42189fe97f024808be24b90468082e48/roles/2aa3a46a15e347ceaebef17caa92ba7c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:15:08.876675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f2a7-5bdc-4fce-a8e7-7bd6a9086175 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.876980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f2a7-5bdc-4fce-a8e7-7bd6a9086175 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.877257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f2a7-5bdc-4fce-a8e7-7bd6a9086175 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.879080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 79/159] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:08.885496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:08.885979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:08.886359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:08.906815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:08.908137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:08.956636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:08.969322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e09d0-125f-4eb4-80bb-92564d49c489 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.040939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 80/160] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:09.055282 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.056769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.057277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.057738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.076345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:09.081677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:09.082537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:09.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:09.085613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 81/161] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:09.091015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.095320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.132633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:09.225142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a15a-fb01-4539-88d5-8323d76c5374 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.244635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4f2e-14bb-491e-b857-e67e2d2588c2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.263912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.264571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.264851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.265126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.277409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-504d78ab-13e9-4ce7-94a7-c1190734b299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:09.282895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:09.283757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:09.284639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 82/162] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:09.285285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de11fdb-1dd6-4116-8937-291d4d5111b6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.285824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de11fdb-1dd6-4116-8937-291d4d5111b6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.285984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de11fdb-1dd6-4116-8937-291d4d5111b6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.287120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 83/163] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:09.292052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.292696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.292877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de11fdb-1dd6-4116-8937-291d4d5111b6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.293053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.293345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.294085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:09.294391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:09.295862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31944938-c51c-4fab-9c00-5f395dfee465 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:09.322074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 81/164] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:09.329330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.329628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.329907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.370948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:09.372627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:09.424975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:09.526446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:09.532358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 82/165] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:09.533230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.543438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de11fdb-1dd6-4116-8937-291d4d5111b6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.612609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.613236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.613516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.613789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.614637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:09.615068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:09.617105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:09.633959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 84/166] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:09.634954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.674538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0b8a-357a-4afe-9581-137e545b0b50 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.717653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.718313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.718645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.718924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] PATH_INFO: `/v3/users/feddce10ac514a48a58f8b731d31a20b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.722259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] RBAC: Authorizing `identity:get_user(user_id=feddce10ac514a48a58f8b731d31a20b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:09.723732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:09.725447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 83/167] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/users/feddce10ac514a48a58f8b731d31a20b => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:09.726749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.747038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:09.878628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d9eb8-cd01-4e1a-ae69-ba7a8e65d01e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:09.919448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:09.920145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:09.920426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:09.920702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:09.921890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:09.923385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:09.925250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 84/168] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:09.925853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:09.949844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42d006-5223-4843-af12-a38bf54b599c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:10.036615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:10.037289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:10.037575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:10.038087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:10.056806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:10.068567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:10.069472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:10.070249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 85/169] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:10.070924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:10.074642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.092204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.126679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460696d-e5b4-42a6-a587-3169c26b31ca None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:10.232661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:10.233640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:10.234133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:10.234622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:10.255288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55e034-2ed3-43a3-8e55-79df50e7cd6e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:10.315213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:10.315854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:10.316155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:10.316412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:10.395118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:10.397082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:10.408651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:10.410118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:10.414190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.540666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 86/170] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:10.541481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:10.601250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.627673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 85/171] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:10.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:10.760151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ab5e2-89ed-4f0e-b76d-c7d693f99212 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:10.808976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b16b2e-a12a-4835-a964-efa8bf53b73b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:10.812722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:10.813434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:10.813721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:10.813998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:10.821021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:10.822484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:10.824315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 86/172] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:10 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:10.825072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:10.837051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.880534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:10.881174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:10.881463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:10.881770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:10.882497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:10.882960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:10.884528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:10.898128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 87/173] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:10.899217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:10.916185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:10.962054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55136a-f379-40a5-bce4-f91b2b292583 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:11.045485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:11.046151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.046435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.046712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.049908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:11.051386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:11.053272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 87/174] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:10 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:11.055723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:11.105986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1401-7d3f-4116-a555-cf2f434b1bd7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:11.169444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:11.169444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.169705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.169977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.170662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:11.170968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:11.172516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:11.184677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 88/175] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:10 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:11.185470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:11.261844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:11.301925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a808f5-9576-4056-8efb-95399d8ec8f6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:11.353943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:11.371173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a906-ab82-4377-98c0-7c1c9ea25ef6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:11.385789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:11.386898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.389006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.389440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] PATH_INFO: `/v3/users/fc52d58399b846da8fd7d4217e87ec87` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.391626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] RBAC: Authorizing `identity:get_user(user_id=fc52d58399b846da8fd7d4217e87ec87)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:11.394042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:11.396105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 88/176] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:11 2022] GET /identity/v3/users/fc52d58399b846da8fd7d4217e87ec87 => generated 275 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:11.402392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:11.430374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:11.432141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.433348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.433951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.435995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=feddce10ac514a48a58f8b731d31a20b) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:11.437319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:11.438950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efe6aa-f419-4b82-8dec-a09fc4b0979e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:11.452541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 89/177] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:11 2022] GET /identity/v3/role_assignments?user.id=feddce10ac514a48a58f8b731d31a20b&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:11.603346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7962-7ed9-4e44-bb50-0040cc6cd9cc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:11.638910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cd32-b804-445e-a8ec-3f352d4ff413 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.639417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cd32-b804-445e-a8ec-3f352d4ff413 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.639860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cd32-b804-445e-a8ec-3f352d4ff413 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.641362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 90/178] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:11.646745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee435-5c21-4ed9-9f82-89e53cf84216 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.647218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee435-5c21-4ed9-9f82-89e53cf84216 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.647668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee435-5c21-4ed9-9f82-89e53cf84216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.678121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:11.679559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.680623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f74ee435-5c21-4ed9-9f82-89e53cf84216 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:11.681216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.681644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.683617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f74ee435-5c21-4ed9-9f82-89e53cf84216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:11.781633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:11.799418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:11.800924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:11.807035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 91/179] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:11.808520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888667-cec5-408b-94f1-e8af1c319fbe None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.808967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888667-cec5-408b-94f1-e8af1c319fbe None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.809400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888667-cec5-408b-94f1-e8af1c319fbe None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.815854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 92/180] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:11.817470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:11.817911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:11.818441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:11.840538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:11.841751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:11.871187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:11.919050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 93/181] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:11.924700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 89/182] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:11.924829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:11.930901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.037732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1e9d2-9151-4924-ada1-3e576f443726 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.045885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.057437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3357-0559-458d-9a15-8f5370db30da None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.079551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.080557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.081052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.081998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.096331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.096984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.097266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.097518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:12.097619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] PATH_INFO: `/v3/projects/d7e512911092497dbabb751ef98f110e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.098901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e512911092497dbabb751ef98f110e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.100436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.102715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 90/183] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:11 2022] GET /identity/v3/projects/d7e512911092497dbabb751ef98f110e => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:12.103478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6bd90a02-25d5-451d-bf7a-89c8a9bf8f6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:12.104570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:12.105439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:12.106222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:12.109764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 94/184] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:12.111221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed351-8f98-4a70-b5e5-5ca3886eac23 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.111710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed351-8f98-4a70-b5e5-5ca3886eac23 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.112208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aed351-8f98-4a70-b5e5-5ca3886eac23 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.113901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 95/185] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:12.115251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aed351-8f98-4a70-b5e5-5ca3886eac23 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.140962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.204029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b5352-bf95-4ffa-97db-9af29f6e8792 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.218592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aed351-8f98-4a70-b5e5-5ca3886eac23 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.238116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.238754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.239038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.239639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.240465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.241961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242edfcf-cf66-48c2-957c-53c510c32909 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.243845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 91/186] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:12.244854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0057-99b2-4af8-86de-f662f1b01f49 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.245138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0057-99b2-4af8-86de-f662f1b01f49 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.245413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0057-99b2-4af8-86de-f662f1b01f49 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.246621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 92/187] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:12.251375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0057-99b2-4af8-86de-f662f1b01f49 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.254102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.254714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.254997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.255275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.256100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=d7e512911092497dbabb751ef98f110e, user.id=fc52d58399b846da8fd7d4217e87ec87) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:12.256412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.258016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e35788-4220-4219-8415-653fd011f0f4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.263769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 96/188] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/role_assignments?user.id=fc52d58399b846da8fd7d4217e87ec87&scope.project.id=d7e512911092497dbabb751ef98f110e&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:12.264997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf90b-4d32-4191-b77b-cf4f215832ba None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.265287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf90b-4d32-4191-b77b-cf4f215832ba None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.265563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf90b-4d32-4191-b77b-cf4f215832ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.285142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3b0bf90b-4d32-4191-b77b-cf4f215832ba None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:12.286477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3b0bf90b-4d32-4191-b77b-cf4f215832ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:12.295596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.372778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 97/189] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:12.374399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.374898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.375336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.392982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0057-99b2-4af8-86de-f662f1b01f49 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.396914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:12.398684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:12.410276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.435145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.436451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.436747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.437076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.437836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:12.438143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.439525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.448798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 93/190] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:12.449891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.479374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 98/191] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:12.480753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.546967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.558802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78fde5-97e0-450d-99c7-3b658e5ea995 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.592895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.593400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a21d-deb0-406f-a9d9-0ef44f486924 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.593508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.593782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.594060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.594740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.596195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.627438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 94/192] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:12 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:12.627438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.631089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.632382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.632672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.632997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.645395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:12.651232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:12.652221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:12.653056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:12.656221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 99/193] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:12.657679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.667979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.750664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ce41d-be20-4f37-8a50-6614186ceeb3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.773064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed32e9-b030-4d03-9f5f-8aa7f585ca38 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.795143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.795794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.796112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.796393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.812153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:12.815423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:12.815671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:12.815791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 95/194] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:12.816402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:12.816628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:12.816904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:12.820692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.823739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.831509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:12.837741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:12.838630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 100/195] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:12.839773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:12.927879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:12.940686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd59f-f69a-42aa-81d5-a547bcff5a7c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:12.971656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93611391-7895-40b3-8006-0b8bd329c6a3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.000127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.001400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.002131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.003029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.004356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:13.005114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.008422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.011076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.011694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.012216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.012320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.020308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 96/196] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.021879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.097058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.103934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.107177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.140860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943e26e-0cc1-4359-89bb-d4b715406539 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.176552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.177181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.177463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.177787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.179420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:13.179420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.180806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.189858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 97/197] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.190591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.196162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.238185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 101/198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:13.243465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.330519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09bb2-16ec-4fd4-8750-e1a6599397c7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.345426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f619a-8322-4339-aa4d-96f8fc07da54 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.361830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.372447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.373357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.373710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.374067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.375371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.377041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.378729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 98/199] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.379584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.391740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.392666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.393414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.393865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] PATH_INFO: `/v3/users/42189fe97f024808be24b90468082e48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.395430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] RBAC: Authorizing `identity:get_user(user_id=42189fe97f024808be24b90468082e48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.398169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.400650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 102/200] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/users/42189fe97f024808be24b90468082e48 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.401827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.444747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.499265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c248-aeda-4ad5-b4c4-3ad01d10e029 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.519434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd134-45eb-4bff-a69e-3970bacbdec1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.543727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.544843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.545663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.546102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.547941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.550211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.552064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 99/201] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.559246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.565589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.566319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.566699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.567071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.626550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.644414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.646016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.666973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc1c2-56f3-4dad-a510-b259f5452fdd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.682638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.701665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.702483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.702904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.703338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.758129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 103/202] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:13.759255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.773089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.774752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.835888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.870268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855761-2756-4bee-9a5e-bc4e598eeb4f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:13.878232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 100/203] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:13.879437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:13.909164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:13.909792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:13.910075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:13.910351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:13.911257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=c99734fcf75741189083c0050a357fcc) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:13.911597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:13.914080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:13.927546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 104/204] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/role_assignments?user.id=c99734fcf75741189083c0050a357fcc&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:13.927546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:13.928899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:14.091662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a5f7-cf1d-4e55-b295-da26534fe262 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:14.095736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629263fc-c9a8-42e6-a13a-8cb994752b21 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:14.150003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:14.150982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.151455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.152184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.153541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:14.155199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:14.158112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 105/205] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:14.163362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:14.164879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:14.167411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.168119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.169239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.170820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:14.175410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:14.179971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 101/206] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:13 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:14.186578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:14.247941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:14.270648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:14.323620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990348-0d53-41b0-a746-8923362a09ed None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:14.341250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79152-ece9-480d-9c65-bed9e0a2e3b2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:14.387680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:14.388356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.388617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.388893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.389702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, scope.project.id=35b4380116c842509fe369b502503cb8, user.id=42189fe97f024808be24b90468082e48) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:14.390012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:14.392612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f85d1-f664-4c26-be04-29a95b557728 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:14.402737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 106/207] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:14 2022] GET /identity/v3/role_assignments?user.id=42189fe97f024808be24b90468082e48&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:14.404792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:14.405418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.405703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.405982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.406788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=c99734fcf75741189083c0050a357fcc) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:14.407103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:14.408703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40766de-c642-49fe-a512-31839cf71a31 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:14.419971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 102/208] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:14 2022] GET /identity/v3/role_assignments?user.id=c99734fcf75741189083c0050a357fcc&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:14.853667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20534d7-07fc-4212-bc3b-31783d2da905 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.853981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20534d7-07fc-4212-bc3b-31783d2da905 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.854340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20534d7-07fc-4212-bc3b-31783d2da905 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.855860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 107/209] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:14.861805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.862255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.862428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.887733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:14.888979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:14.932277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de0ab-02fa-44ec-bbd3-76e6906d4ef9 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.932578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de0ab-02fa-44ec-bbd3-76e6906d4ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.932840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de0ab-02fa-44ec-bbd3-76e6906d4ef9 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.934201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 108/210] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:14.939415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:14.939700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:14.940000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:14.966101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:14.967408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:14.978017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 103/211] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:15.000981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.027698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:15.040018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 109/212] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:15.064724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.121377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8340bd-3cd5-43aa-88e7-a4d71df1e150 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.139471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:15.163364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.164348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.164805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.165301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.178112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:15.184131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:15.185068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:15.185842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 104/213] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:15.193733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb4227-0538-47ab-946e-0de7ef8c9d6e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.194003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.234281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.235253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.235720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.236124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.237930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:15.239511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:15.246843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 110/214] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:15.248334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.305506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:15.327468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467515a-8190-45f1-a359-a82c2ab67967 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.375642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.376299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9989-1b7a-4a9e-98e3-af8bb81643dd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.376713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.377211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.377712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.379643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:15.380174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:15.381746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:15.393022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 105/215] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:15.398464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.411640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:15.423375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.424957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.425495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.425987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.426924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:15.429429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:15.467946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 111/216] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:15 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:15.536532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681847ac-df6d-46f1-966d-2e15d90b3f80 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.577491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.578534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.579019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.579489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] PATH_INFO: `/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.596890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c9d7ef8cb54f6f995b2ade51efea20)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:15.598746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:15.600981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 106/217] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:15 2022] GET /identity/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:15.611623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.722205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7f3d-7cbc-4cf6-83f1-67b0b818f2a2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:15.758153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:15.759142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:15.759427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:15.759707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:15.823373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:15.840716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:15.842531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99db3f-b6f7-4697-b069-68cc94ddac4c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:15.952426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 112/218] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:15.958286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:15.963779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:16.077520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6acd94-964c-4640-a07d-16c6b74f7657 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.102920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc91dc-91c2-447c-bc00-24358ed4452f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.103517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc91dc-91c2-447c-bc00-24358ed4452f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.104039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc91dc-91c2-447c-bc00-24358ed4452f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.105831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 113/219] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:16.110952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.111510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.112027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.121170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:16.121835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.122123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.122400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.126981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:16.128799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:16.130505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 107/220] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:15 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:16.133161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:16.134568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:16.136861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:16.221056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:16.256641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f837-f8aa-40fe-a56d-ec3a7792dd6d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.280263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 114/221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:16.302358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:16.302567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:16.302999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.303387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.304128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.304937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=a7c9d7ef8cb54f6f995b2ade51efea20) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:16.305250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:16.306828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:16.319876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:16.321902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 108/222] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/role_assignments?user.id=a7c9d7ef8cb54f6f995b2ade51efea20&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:16.411034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eba632-cc2e-4abf-82e9-43f91e6a225f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.448462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:16.450216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.450696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.451192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.463549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1432e282-642c-4763-bd84-687e4e909c84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:16.468903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:16.469764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:16.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:16.472886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 115/223] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:16.479565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:16.616710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86056251-b154-4959-8ef1-c823611a95a3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.625913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdf1cb-6b19-416c-9efd-33827708c2f5 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.626516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdf1cb-6b19-416c-9efd-33827708c2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.627009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdf1cb-6b19-416c-9efd-33827708c2f5 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.629215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 116/224] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:16.634844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.635419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.635908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.660784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:16.665349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:16.692584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:16.694402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.694402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.694402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.694894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:16.695170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:16.696643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:16.699368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:16.707787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 109/225] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:16.716734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:16.756366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 117/226] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:16.787353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:16.799111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:16.887627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a95fa-0039-4434-bce1-a63abcdb2d3b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.958728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933aad-84d2-4e8d-b098-f3c7df3cb6d0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:16.977504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:16.978158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:16.978436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:16.978706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:16.981296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:16.982742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:16.991315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 110/227] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:16.993435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.037446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.038162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.038447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.038724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.050680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:17.056209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:17.057088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:17.058428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 118/228] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:17.068822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.107205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:17.156779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:17.173073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c7e3-a8a3-4248-a201-368e749c977d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.201639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab2e8-1ac0-436b-aa67-1e00ba5663b9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.239544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.240150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.240434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.240731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.241005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.241860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.242337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.242805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.243722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:17.244250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.244563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.246077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.246183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.248111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 111/229] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:16 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:17.253211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.256073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 119/230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:17.262450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.432520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811a084-8c0c-46d1-a25d-f6b65d86d4c2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.448605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb86e-13ae-4acc-af3a-33780ed85312 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.477044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:17.487805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.489387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.490132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.491328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.492926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.495152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.497297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 120/231] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:17 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:17.502813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.532556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.533353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.533529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.533815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/c99734fcf75741189083c0050a357fcc/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.540904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, user_id=c99734fcf75741189083c0050a357fcc, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.542038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.548202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:17.598067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 112/232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:17 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/c99734fcf75741189083c0050a357fcc/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:17.620960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479eac-0f6b-4e79-89e4-526a5b2e9c44 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.674203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.675213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.675689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.676197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.680688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.682461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61a12a-21e9-46d6-9dd8-7df9b0ff606c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.684442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 121/233] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:17 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:17.690374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:17.806538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135047f-410e-4436-b298-e644fdd933e2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:17.845854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:17.846881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:17.847376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:17.847867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/c99734fcf75741189083c0050a357fcc/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:17.851492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, user_id=c99734fcf75741189083c0050a357fcc, role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:17.853316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d41e4-1268-49eb-89e6-f842af760532 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:17.877517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 113/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:17 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/c99734fcf75741189083c0050a357fcc/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:18.120128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee191e-aee2-4c33-9f80-503ad76002b0 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.120417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee191e-aee2-4c33-9f80-503ad76002b0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.120833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee191e-aee2-4c33-9f80-503ad76002b0 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.122521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 122/235] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:18.127908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.128345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.128534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.151396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:18.152636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:18.209924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3561-600c-4c6a-92f0-2441babffb6f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.210250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3561-600c-4c6a-92f0-2441babffb6f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.210572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3561-600c-4c6a-92f0-2441babffb6f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.212088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 123/236] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:18.217071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.217465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.217746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.223107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:18.239512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:18.241237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:18.250124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 114/237] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:18.288867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.313117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 124/238] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:18.316199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:18.339031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.441460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0cb2d-a5e7-4ab2-be64-e522855408ab None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:18.472166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec65433-1ef4-40f3-84c0-5069a804cde4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:18.500633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:18.501288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.501561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.501861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.508135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:18.508594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:18.508851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.509135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.509416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.510058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:18.511963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 125/239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:18.517312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.520478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:18.526120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:18.526998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:18.527790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 115/240] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:18.533148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.584531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:18.632201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:18.638940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766ed31-7572-4f9f-8dd1-f0ec7a72f1f4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:18.674427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:18.675052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.675338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.675620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.676383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:18.676693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:18.678080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:18.691622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 116/241] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:18.692482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9a45-951b-452e-9d4b-b721eb4d5fee None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:18.692814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.759238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:18.759892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.760178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.760455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.761156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:18.762607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:18.812783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 126/242] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:18 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:15:18.855534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616deab-42a3-435a-9f3b-130eda70d47e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:18.872262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:18.892241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:18.892909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.893195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.893472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] PATH_INFO: `/v3/users/87886919f54e42018687573a95316f30` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.910025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] RBAC: Authorizing `identity:get_user(user_id=87886919f54e42018687573a95316f30)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:18.911501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6ba0c-7d5d-4a80-af3a-47a180558351 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:18.918568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 117/243] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:18 2022] GET /identity/v3/users/87886919f54e42018687573a95316f30 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:18.920076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:18.996293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59561-69f1-4d70-8442-66a4eb5df656 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:18.996602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59561-69f1-4d70-8442-66a4eb5df656 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:18.996847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee59561-69f1-4d70-8442-66a4eb5df656 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:18.998517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 118/244] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:19.003470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.003755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.004069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.032129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:19.033383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:19.093578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953171-2b31-469a-a544-8f71143823a9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.117129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.120370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 119/245] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:19.130968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.131752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.132269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.132753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.148897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.222925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:19.224604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:19.247931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.256091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66819f0c-d9c2-4f76-98ff-6a4e4e86ea8b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.304837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.305547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.306094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.306376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.320889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:19.327736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:19.328717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:19.329502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:19.330103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 120/246] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:19.336042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.363350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 127/247] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:19.377320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.401784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.481559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93356c2-97b8-4567-a0ca-3977e135492f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.502812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfa92b-fc73-4d7d-800e-b0b845d85295 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.527889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.529003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.529418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.529713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.530450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:19.530759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:19.532334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:19.544183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 121/248] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:19.546838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.547497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.547780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.548089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.549595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.552226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:19.554570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:19.557622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 128/249] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:19.562926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.573921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.681710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.687973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f811-8a36-4519-8343-b5ce2a3ca3ee None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.714788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d532c-4a3f-4398-8f54-63846d0b82b4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.750538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.751108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.751389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.751658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] PATH_INFO: `/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.754196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c9d7ef8cb54f6f995b2ade51efea20)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:19.755744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:19.759090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 122/250] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:19.763272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:19.780473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.781096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.781376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.781648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.782454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=87886919f54e42018687573a95316f30) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:19.782762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:19.784252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:19.794206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 129/251] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/role_assignments?user.id=87886919f54e42018687573a95316f30&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:19.922766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:19.942771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f0ee-85e5-4a6b-acf6-451a975b1222 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:19.996825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:19.997496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:19.997782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:19.998060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:19.999274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:20.000817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43fbaa-66ef-4d29-9ea8-9b0f0df2417f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:20.002752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 123/252] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:19 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:20.012918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:20.163912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44c195-99db-40e2-8b6f-7d1dbf5c39ff None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:20.198933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:20.199617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.200192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.200474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/a7c9d7ef8cb54f6f995b2ade51efea20/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.202438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, user_id=a7c9d7ef8cb54f6f995b2ade51efea20, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:20.203982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a9b1-890f-4fc7-8482-17604947a8f4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:20.223170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 130/253] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:20 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/a7c9d7ef8cb54f6f995b2ade51efea20/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:20.407128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769c53-9a05-4c52-9f4d-262220e3b9b8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.407677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769c53-9a05-4c52-9f4d-262220e3b9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.407677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30769c53-9a05-4c52-9f4d-262220e3b9b8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.409350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 124/254] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:20.414799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.415088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.415369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.441311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:20.442529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:20.444867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:20.528143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 131/255] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:20.552887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30769c53-9a05-4c52-9f4d-262220e3b9b8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:20.667187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30769c53-9a05-4c52-9f4d-262220e3b9b8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:20.714611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:20.715253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.715533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.715807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.734762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ca6605c8-3873-468c-a1f8-f9dde0f0748c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:20.741441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:20.742310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:20.743494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 125/256] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:20.748670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:20.813484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3fc0b-802a-481b-8d2a-801b7ffb9283 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.813802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3fc0b-802a-481b-8d2a-801b7ffb9283 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.814035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3fc0b-802a-481b-8d2a-801b7ffb9283 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.816432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 126/257] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:20.821378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:20.822111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:20.822391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:20.847878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:20.848715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:20.868469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:20.942113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fdeb-2ca1-4930-8cbf-b165bd048906 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:20.993152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 127/258] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:21.000142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.000768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.001048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.001319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.002045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:21.002345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:21.003791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:21.009188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:21.013576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 132/259] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:21.019206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.034666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.194585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73971981-04ba-4e32-931f-4abb5a494c0a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:21.218565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e87f35-89aa-4089-891d-832415b82668 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:21.260305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.261005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.261288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.261566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.262773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:21.266245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:21.267856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 128/260] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:21.275545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.286129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.286830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.287186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.287526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.305153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:21.305665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:21.314229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:21.315202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:21.316032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:21.317225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 133/261] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:21.322567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.440850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:21.445518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156b878-cf4a-40f0-8623-8f9d7b43bb09 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:21.525467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.526105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.526392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.526672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.587326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a392cd-7831-490e-a4f2-224efa122dfb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:21.660673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:21.662351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:21.681075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.681710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.681993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.682269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.682989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:21.683291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:21.683702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:21.684698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:21.703610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 134/262] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:21.709867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.899191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970f179-60ee-4249-a733-c29b9d1aa72b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:21.915552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 129/263] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:21.921089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:21.941808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:21.959436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:21.959436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:21.959436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:21.959436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] PATH_INFO: `/v3/users/c99734fcf75741189083c0050a357fcc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:21.959885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] RBAC: Authorizing `identity:get_user(user_id=c99734fcf75741189083c0050a357fcc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:21.959885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:21.968283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 135/264] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/users/c99734fcf75741189083c0050a357fcc => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:21.969340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.083780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17b8a-dad8-4f9c-aefb-a5d42b9ce68a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.095280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.127281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4284f-77e2-47ec-8648-4cf967afbfa9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.137483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.138148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.138431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.138706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.144634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.146127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.147912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 130/265] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:22.156842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.182481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.183119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.183406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.183684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.259931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.261349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.264121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.304800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e441c0-adeb-42f9-b195-b9606c597e55 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.366954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 136/266] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:22.372552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.372982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.373875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.374464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.374934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.376050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=c99734fcf75741189083c0050a357fcc) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:22.376575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.378323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83f825-0968-4cd5-b8cb-90a437cb23d7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.391768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 131/267] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:22 2022] GET /identity/v3/role_assignments?user.id=c99734fcf75741189083c0050a357fcc&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:22.440091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ed41f-06ca-4f4b-82cd-8fb215273bd5 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.440411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ed41f-06ca-4f4b-82cd-8fb215273bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.440658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ed41f-06ca-4f4b-82cd-8fb215273bd5 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.443502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 132/268] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:22.448037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e4ee-3df8-47ee-a721-983a7aa7e6c5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.448621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e4ee-3df8-47ee-a721-983a7aa7e6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.448621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e4ee-3df8-47ee-a721-983a7aa7e6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.468050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c4a9e4ee-3df8-47ee-a721-983a7aa7e6c5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:22.469435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c4a9e4ee-3df8-47ee-a721-983a7aa7e6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:22.478480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036bb27-1fd4-4e56-8001-ca182969c88a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.492248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.518717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.526469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.527136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.527421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.528094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.530572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.532141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db793c04-e131-4b29-826d-f65fbea86097 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.533916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 137/269] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:22 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:22.536245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b270f-976c-4234-b8f8-59abcf531b53 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.536535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b270f-976c-4234-b8f8-59abcf531b53 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.536821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b270f-976c-4234-b8f8-59abcf531b53 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.539088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 138/270] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:22.540777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b270f-976c-4234-b8f8-59abcf531b53 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.587118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 133/271] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:22.588624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.588920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.589205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.624472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:22.626265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:22.714358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 134/272] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:22.715246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.721708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b270f-976c-4234-b8f8-59abcf531b53 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.774542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.776064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.776550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.777023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.778448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=c99734fcf75741189083c0050a357fcc) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:22.778964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.781350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.790163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 139/273] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:22 2022] GET /identity/v3/role_assignments?user.id=c99734fcf75741189083c0050a357fcc&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:22.791283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.827136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.855481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925f3e-3e5a-43bd-be4e-f4ef1ed106cc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.857504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:22.908276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.909276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.909564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.909844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.912897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:22.914955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645139a-959c-4f18-9214-3947a35f6f60 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:22.917313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 135/274] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:22.918731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17f4a-3607-4d86-b8de-ce80bcce5527 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:22.919024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17f4a-3607-4d86-b8de-ce80bcce5527 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:22.924160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17f4a-3607-4d86-b8de-ce80bcce5527 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:22.926001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 136/275] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:22.927559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc17f4a-3607-4d86-b8de-ce80bcce5527 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:22.941736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020360-9d73-4c1a-b72e-d033dbb2bc41 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:22.998579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:22.999669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.000246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.000749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.016712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-263acb70-e41a-492c-8df3-8734baea50ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:23.022967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:23.023928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:23.024977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:23.027667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 140/276] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:23.029014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.029435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.029864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.049829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:23.051006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:23.060885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc17f4a-3607-4d86-b8de-ce80bcce5527 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.104280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.104953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.105234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.105506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.106190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.107670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.133382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 137/277] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:22 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:23.135095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.151288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 141/278] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:23.172128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.180933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.207191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.268467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6fad-9da4-4743-a8b3-e1d4f36ba3a6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.309567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.310115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.310399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.310672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.311386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:23.311686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.313121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.324023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16078a5c-9dea-4ff5-ad67-0ddd845ad4ca None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.326377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 138/279] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:23.327529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.373280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.373934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.374216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.374807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.386638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:23.391766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:23.392888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:23.393740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 142/280] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:23.399612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.431229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.446662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c79a2-a81e-4285-8c83-8d6739318536 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.493221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.497945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.499240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.499718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.500213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] PATH_INFO: `/v3/users/9176fb4500454829a6061fae217aae4b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.515205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] RBAC: Authorizing `identity:get_user(user_id=9176fb4500454829a6061fae217aae4b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.517036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c87865-e04d-4ecb-9ec0-0e9ce5c8c161 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.519334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 139/281] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/users/9176fb4500454829a6061fae217aae4b => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:23.520883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042511-2ef0-4076-ac71-d05b8611992e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.521400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042511-2ef0-4076-ac71-d05b8611992e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.522417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042511-2ef0-4076-ac71-d05b8611992e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.523913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 140/282] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:23.528696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21042511-2ef0-4076-ac71-d05b8611992e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.531227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925cba-861d-4d50-bc20-53eb32f66bb1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.569403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.569948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.570234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.570513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.571246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:23.571553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.573001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213638d3-f50d-4e55-84f6-cd29c798584a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.583764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 143/283] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:23.589312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.589312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.589312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.608413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:23.609790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:23.657294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21042511-2ef0-4076-ac71-d05b8611992e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.690727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 144/284] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:23.691456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.701303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.702216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.702672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.703120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.704240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, user.id=9176fb4500454829a6061fae217aae4b, scope.system=all) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:23.705597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.707340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.714478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 141/285] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/role_assignments?user.id=9176fb4500454829a6061fae217aae4b&scope.system=all&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:23.731751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.755269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.786196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:23.825621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714a19-521b-4da9-a192-a888dab92ca0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.889421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.890090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.890368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.890643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] PATH_INFO: `/v3/users/87886919f54e42018687573a95316f30` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.892974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] RBAC: Authorizing `identity:get_user(user_id=87886919f54e42018687573a95316f30)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:23.894457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:23.898006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 145/286] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/users/87886919f54e42018687573a95316f30 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:23.904679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:23.944804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17e10-a4b0-40d5-a590-6c921171fb3f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:23.983602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:23.984558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:23.985410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:23.985828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:23.997119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:24.003125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:24.004398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:24.005331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:24.008289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 142/287] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:24.020831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:24.103450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7be06-451a-4949-aaa2-1884d4e9d9af None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:24.152690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:24.160698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:24.169742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:24.170386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:24.170665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:24.170938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:24.172269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:24.173719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:24.182901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 146/288] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:23 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:24.184535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:24.216110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fd8e1-ef34-43dd-bbb9-7a12f9f77123 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:24.278164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:24.278782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:24.279062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:24.279339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:24.280072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:24.280379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:24.281755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:24.293697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 143/289] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:24.300077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:24.351855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4489c-78bf-49ed-b908-1c4f852af23c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:24.407168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:24.407760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:24.408065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:24.408340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/87886919f54e42018687573a95316f30/roles/2aa3a46a15e347ceaebef17caa92ba7c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:24.411004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, user_id=87886919f54e42018687573a95316f30, role_id=2aa3a46a15e347ceaebef17caa92ba7c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:24.413040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:24.441687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 147/290] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:15:24 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/users/87886919f54e42018687573a95316f30/roles/2aa3a46a15e347ceaebef17caa92ba7c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:24.482291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2ae81-5211-417f-9a89-afa5a64116b1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:24.526182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:24.532149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:24.532912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:24.533194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:24.533466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] PATH_INFO: `/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:24.540847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] RBAC: Authorizing `identity:get_user(user_id=a7c9d7ef8cb54f6f995b2ade51efea20)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:24.542380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:24.544538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 144/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:24 2022] GET /identity/v3/users/a7c9d7ef8cb54f6f995b2ade51efea20 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:24.550131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:24.636286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:24.691517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ba47-ce88-4c9d-be28-7863b6bebb6b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:24.741566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:24.742204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:24.742485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:24.742762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:24.829013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:24.830495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:24.893697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:24.947597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 148/292] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:24.956938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:25.115932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf0b8-d368-484b-8fdf-15932c18983f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:25.168138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:25.169755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.169755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.169755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.170526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:25.172544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443e901-6ff3-43ab-aa7a-25bd4c965f18 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:25.173958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 145/293] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:24 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:25.180373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:25.334147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf973e-fa74-411d-80ba-18611e695794 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:25.369741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:25.370739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.371223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.371701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.372725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=a7c9d7ef8cb54f6f995b2ade51efea20) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:25.373255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:25.374900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddb00e-1686-44b3-8cbd-1818fec487dd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:25.382009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 149/294] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:25 2022] GET /identity/v3/role_assignments?user.id=a7c9d7ef8cb54f6f995b2ade51efea20&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:25.679888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f254b-cb8d-43ff-b0bb-be5a139a859f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.680399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f254b-cb8d-43ff-b0bb-be5a139a859f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.680474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f254b-cb8d-43ff-b0bb-be5a139a859f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.682032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 146/295] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:25.687349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.687898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.688394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.695597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bc49-6f7c-4f3f-9784-8d19706cac75 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.695937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bc49-6f7c-4f3f-9784-8d19706cac75 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.696179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bc49-6f7c-4f3f-9784-8d19706cac75 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.697766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 147/296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:25.705382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:25.705742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:25.706090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:25.711050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:25.712516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:25.730242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:25.731437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:25.734660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:25.802055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 150/297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:25.808240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:25.816896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 148/298] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:25.829021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:25.847274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:25.956414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf245e-82bd-4684-92a5-4f6f4d254aab None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:25.979040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf88e-328b-4524-8357-3d9a08076064 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.002288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.003582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.004378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.004859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.006924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.009566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.012635 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 151/299] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:26.018278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.019726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.020562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.020847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.021123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.034044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:26.040098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:26.041001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:26.041822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:26.046332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 149/300] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:26.047294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.053719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.119483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.152025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819cec-4d76-4849-926d-165ea23fcb32 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.163583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bfcbd-7cdf-4ce0-a5ac-03136dd41171 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.187468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.188080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.188366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.188802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] PATH_INFO: `/v3/groups` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.189302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.190748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343d082-a380-4b09-9093-b9c0371b73b8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.200632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.201479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.201760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.202037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.202734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:26.203039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.204446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.210250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 152/301] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 15:15:26 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:15:26.212069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3710-fae3-4518-a743-0bb411a33921 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.212522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3710-fae3-4518-a743-0bb411a33921 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.212971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 150/302] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:26.218036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f3710-fae3-4518-a743-0bb411a33921 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.218036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 153/303] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:26.221061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.223301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.223588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.223884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.245917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:26.247139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:26.338150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78d743-dd8f-4763-b683-8cc12a85ad45 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.346125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 154/304] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:26.363790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.369448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.381296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.382234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.382519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.382794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] PATH_INFO: `/v3/users/b778f138902a46b4aebd49ba85b4a454` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.385760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.398832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] RBAC: Authorizing `identity:get_user(user_id=b778f138902a46b4aebd49ba85b4a454)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.400405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.402395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 151/305] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/users/b778f138902a46b4aebd49ba85b4a454 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:26.408625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.501581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48865e-0b75-48ff-9d67-106fe666c4d2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.520980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5c69-9b63-4365-ac37-9fa61f978101 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.549377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.550443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.551002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.551286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.557768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.558486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.558833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.559179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.560114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, user.id=b778f138902a46b4aebd49ba85b4a454, scope.system=all) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:26.560488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.561984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.563427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:26.567776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 152/306] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/role_assignments?user.id=b778f138902a46b4aebd49ba85b4a454&scope.system=all&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:26.569665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:26.570528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:26.571661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 155/307] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:26.578714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.640590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.696180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1c2c7-f3ea-4542-8b9d-c4910f9a0209 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.736000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.736623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.736909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.737188 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.737903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:26.738213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.739597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.748650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 153/308] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:26.754062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.830930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:26.865100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e81c3-6cde-4dad-abdc-c024489a58ca None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:26.902216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:26.903130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.903584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.904041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] PATH_INFO: `/v3/users/9176fb4500454829a6061fae217aae4b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.905375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] RBAC: Authorizing `identity:get_user(user_id=9176fb4500454829a6061fae217aae4b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:26.906953 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c572015-460c-459d-8939-14e7eca32a2a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:26.909051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 156/309] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:26 2022] GET /identity/v3/users/9176fb4500454829a6061fae217aae4b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:26.917155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:26.948924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff59ffd-8546-4842-a137-8c1b20c57336 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.949211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff59ffd-8546-4842-a137-8c1b20c57336 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.949489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff59ffd-8546-4842-a137-8c1b20c57336 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.951223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 157/310] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:26.957304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:26.957870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:26.957870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:26.978004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:26.979183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:27.028152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c071-22d5-4e39-be59-85dfd6939846 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:27.030610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:27.056030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 158/311] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:27.068942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:27.069607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:27.069891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:27.070166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] PATH_INFO: `/v3/system/users/9176fb4500454829a6061fae217aae4b/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:27.071696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9176fb4500454829a6061fae217aae4b, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:27.073139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:27.079298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:27.087149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 154/312] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 15:15:26 2022] PUT /identity/v3/system/users/9176fb4500454829a6061fae217aae4b/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:27.169031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:27.204223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ff33-ff2f-4d5f-b604-7203c1408b29 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:27.244493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:27.245139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:27.245415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:27.245684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:27.256610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:27.261589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:27.262454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:27.263253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 159/313] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:27.269174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:27.372922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3289-d812-4474-b9a2-accd9e0f5e54 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:27.407548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:27.408253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:27.408534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:27.408802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:27.409509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:27.409810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:27.411184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:27.420287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 155/314] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:27.425827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:27.531860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0b74-094b-417e-8fbb-318634ef8878 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:27.560725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:27.568995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:27.569644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:27.569928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:27.570204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] PATH_INFO: `/v3/users/87886919f54e42018687573a95316f30` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:27.572567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] RBAC: Authorizing `identity:get_user(user_id=87886919f54e42018687573a95316f30)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:27.574219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:27.576366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 160/315] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:27 2022] GET /identity/v3/users/87886919f54e42018687573a95316f30 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:27.581961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:27.616611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:27.690132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354fb0-a882-4c34-997d-75c7596bcbef None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:27.727864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:27.729309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:27.729787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:27.730280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:27.802596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:27.805119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:27.840281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:27.909779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 156/316] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:27.915618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:28.023766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2215a-a437-4554-b7a6-8b3d18b0d0f0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:28.058753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:28.059398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.059675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.059974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.061128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:28.062569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aadd9-8d71-49d7-9bbe-9f0c6499a82e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:28.064252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 161/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:27 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:28.070981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:28.173267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d57ee-22ff-4cee-b9f2-54eb8352ac66 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:28.205130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:28.210768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:28.211387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.211664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.211996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.212834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576, user.id=87886919f54e42018687573a95316f30) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:28.213205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:28.215075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651367f-9a61-468c-a28a-ca445821ad62 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:28.221727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 157/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:28 2022] GET /identity/v3/role_assignments?user.id=87886919f54e42018687573a95316f30&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:28.406039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2811bd2-92d1-4b77-a1a0-ecb4219a0027 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.406651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2811bd2-92d1-4b77-a1a0-ecb4219a0027 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.407124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2811bd2-92d1-4b77-a1a0-ecb4219a0027 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.408693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 162/319] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:28.414675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e73c-82cd-4e16-a918-c1f1761d1be8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.414971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e73c-82cd-4e16-a918-c1f1761d1be8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.415234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e73c-82cd-4e16-a918-c1f1761d1be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.434429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85b9e73c-82cd-4e16-a918-c1f1761d1be8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:28.435592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85b9e73c-82cd-4e16-a918-c1f1761d1be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:28.511925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 158/320] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:28.528644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2811bd2-92d1-4b77-a1a0-ecb4219a0027 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:28.596206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:28.631389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea611-3985-4195-a13f-012a47f6ed50 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.631671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea611-3985-4195-a13f-012a47f6ed50 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.631990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea611-3985-4195-a13f-012a47f6ed50 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.633339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 159/321] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:28.636326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2811bd2-92d1-4b77-a1a0-ecb4219a0027 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:28.638379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.639371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.639371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.657757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:28.658928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:28.673350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:28.673989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.674265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.674535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.675665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:28.677141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:28.678785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 163/322] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:28.684790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:28.737002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 160/323] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:28.758146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:28.789582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f05002-6158-46c2-a185-1ecaee391714 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:28.817480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:28.825052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:28.825668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.825947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.826219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] PATH_INFO: `/v3/groups` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.826882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:28.828346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:28.832724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:28.847398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 164/324] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 15:15:28 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:15:28.869115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a501-cddc-4a45-8fdb-bce982bf4159 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:28.909545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:28.910207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:28.910489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:28.910760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:28.923377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:28.928610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:28.929523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:28.930350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:28.932221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 161/325] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:28.935906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.049760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595c589-40e0-4996-a000-f1e5c7a121ad None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.086909 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.087525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.087806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.088115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.088841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:29.089148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:29.090532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517015-4fa1-4f39-8c9f-2e9efe247e1a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:29.099209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 165/326] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:29.104674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.154281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb076-e413-4649-87dd-559905ef2c12 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.154992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb076-e413-4649-87dd-559905ef2c12 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.155473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb076-e413-4649-87dd-559905ef2c12 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.157070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 166/327] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:29.163081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.163367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.163640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.183086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:29.184686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:29.236866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df653c17-d8d2-49e8-93cf-f3331752c626 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.251865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:29.253965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:29.278879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 167/328] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:29.293633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.294289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.294574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.294849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] PATH_INFO: `/v3/users/b778f138902a46b4aebd49ba85b4a454` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.296135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] RBAC: Authorizing `identity:get_user(user_id=b778f138902a46b4aebd49ba85b4a454)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:29.297587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:29.299405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 162/329] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:29 2022] GET /identity/v3/users/b778f138902a46b4aebd49ba85b4a454 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:29.303127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.305365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.416958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8c1b3-937b-4904-8673-cbf3f80c9074 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.445448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d00d12-83a4-4c08-9813-1143576d5400 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.463061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.463718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.464033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.464315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.476724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:29.483155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:29.484127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:29.484910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:29.486046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 168/330] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:29.491889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.498616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.499599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.500128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.500652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] PATH_INFO: `/v3/system/users/b778f138902a46b4aebd49ba85b4a454/roles/2aa3a46a15e347ceaebef17caa92ba7c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.503066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b778f138902a46b4aebd49ba85b4a454, role_id=2aa3a46a15e347ceaebef17caa92ba7c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:29.505172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:29.515969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 163/331] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 15:15:29 2022] PUT /identity/v3/system/users/b778f138902a46b4aebd49ba85b4a454/roles/2aa3a46a15e347ceaebef17caa92ba7c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:29.552039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:29.644158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c39b69-ddd3-4c91-b313-f794249d89b5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.689075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.689779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.690212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.690720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.691772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:29.692113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:29.693606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:29.706090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 169/332] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:29.711861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:29.784091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:29.844905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae12a3f-75be-4db1-a889-8fa57c5f3991 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:29.880270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:29.881500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:29.881782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:29.882053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] PATH_INFO: `/v3/users/9176fb4500454829a6061fae217aae4b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:29.883470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] RBAC: Authorizing `identity:get_user(user_id=9176fb4500454829a6061fae217aae4b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:29.886351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23991c56-27dd-4aeb-a7a5-715546210359 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:29.888483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 164/333] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:29 2022] GET /identity/v3/users/9176fb4500454829a6061fae217aae4b => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:29.894064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:30.053280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba01000-2607-4412-86fb-935001f33f45 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:30.064375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:30.099617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:30.100261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:30.100542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:30.100817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:30.101609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] RBAC: Adding query filter params (role.id=63ebc483a8af4b2584cc1a6f03e1625e, user.id=9176fb4500454829a6061fae217aae4b, scope.system=all) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:30.102046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:30.103479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e1f95-3a4a-4058-8678-eae2e7bd5ab7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:30.122212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 170/334] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 15:15:29 2022] GET /identity/v3/role_assignments?user.id=9176fb4500454829a6061fae217aae4b&scope.system=all&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:31.334739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ece-0757-4bf4-ab8e-44b6d6cd990c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.335222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ece-0757-4bf4-ab8e-44b6d6cd990c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.335222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4ece-0757-4bf4-ab8e-44b6d6cd990c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.336896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 165/335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:31.342471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922648-2363-42be-9ed4-cee609e2f780 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.342893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922648-2363-42be-9ed4-cee609e2f780 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.343066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922648-2363-42be-9ed4-cee609e2f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.363698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fb922648-2363-42be-9ed4-cee609e2f780 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:31.364978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fb922648-2363-42be-9ed4-cee609e2f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:31.452199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 171/336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:31.488525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4ece-0757-4bf4-ab8e-44b6d6cd990c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:31.533251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc06-2a50-481b-be9c-d23c841ab95e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.533573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc06-2a50-481b-be9c-d23c841ab95e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.533814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc06-2a50-481b-be9c-d23c841ab95e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.535177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 172/337] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:15:31.547575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.548204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.548672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.581337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:31.582891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:31.633194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4ece-0757-4bf4-ab8e-44b6d6cd990c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:31.652268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:31.675488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:31.676221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.676506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.676778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.692475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:31.702066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:31.703019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:31.704304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 166/338] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:31.717286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:31.720219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 173/339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:31.760696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:31.812686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:31.846838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142894-c43c-4281-b106-c826b8218b1b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:31.888349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:31.888954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.889234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.889508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:31.890221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:31.890522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:31.892424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:31.901369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 167/340] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:31.909207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:31.936366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77588fad-aed9-42aa-b932-66b1aa49317b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:31.952279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:31.986399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:31.987350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:31.987766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:31.988220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.004685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:32.018069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:32.019161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:32.020103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:32.026587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 174/341] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:32.027424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a522806-a5cf-4f90-99be-55aae756f602 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.027865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:32.135235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:32.136892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.137319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.137769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.172547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.201097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9711-4225-437d-a040-1003b768bdb4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.256161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:32.258270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454208ee-e761-494c-87ec-491f33b50376 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:32.272878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:32.273797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.274275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.274738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.275645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:32.276180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:32.277765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99bf2b-b2d6-4afa-beb0-b42d9fa00080 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:32.294433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.295181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 175/342] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:32.296506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785ecfe-54c2-4c6a-9927-bec17a89364e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.296802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785ecfe-54c2-4c6a-9927-bec17a89364e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.299176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785ecfe-54c2-4c6a-9927-bec17a89364e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.299176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 176/343] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:32.305641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785ecfe-54c2-4c6a-9927-bec17a89364e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:32.433117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 168/344] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:32.435087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.435590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.436095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.455534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:32.457155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:32.489152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785ecfe-54c2-4c6a-9927-bec17a89364e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.561161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:32.562169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.562974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.564287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.577758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 169/345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:32.581271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:32.601797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.676232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.694353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:32.701259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:32.715896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153e4d0-7353-4944-8d23-753f41f1418f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.760918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:32.761931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.762362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.762862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.772225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:32.774045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:32.776128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 170/346] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:32 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:32.777152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:32.835900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 177/347] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:32.836765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:32.890117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.916097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ba627-0f2b-49ef-8e29-a74726606537 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.961803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd0d3-2142-4404-8957-a4fb52aa4247 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:32.976570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:32.977223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:32.977504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:32.977775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:32.980023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:32.991624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8ea1573e-cbb9-4fb0-a3e2-9ca0678dc2d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:32.996796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:32.997968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 15:15:32.998896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:33.000808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 171/348] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:33.002120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efa7fd-577b-41c2-8661-64ac220bdd2f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.002545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efa7fd-577b-41c2-8661-64ac220bdd2f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.002976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efa7fd-577b-41c2-8661-64ac220bdd2f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.004438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 172/349] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:33.005261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efa7fd-577b-41c2-8661-64ac220bdd2f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.007634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.008309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.008669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.008879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.019128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.021009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.022994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 178/350] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:32 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.024030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.152017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efa7fd-577b-41c2-8661-64ac220bdd2f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.165700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cb44b-3dd3-4d86-9c4a-df5e705e9911 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.180844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:33.196640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.197460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.197812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.198189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.199449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.201470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144c9e5-a902-446b-9c2f-78f71ff3474f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.203495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 173/351] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.204992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d790c-b033-4a8b-b696-f66374b70ef5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.205379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d790c-b033-4a8b-b696-f66374b70ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.205838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d790c-b033-4a8b-b696-f66374b70ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.208396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.209756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.210183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.210613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.211347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:33.211659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.213276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.223991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 179/352] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.228415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.235709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8f5d790c-b033-4a8b-b696-f66374b70ef5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:33.235709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8f5d790c-b033-4a8b-b696-f66374b70ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:33.320128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:33.335593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 174/353] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:33.337753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002eacb-f0c6-4533-8955-770b91faabf4 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.337753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002eacb-f0c6-4533-8955-770b91faabf4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.337753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002eacb-f0c6-4533-8955-770b91faabf4 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.339120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 175/354] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:33.347861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002eacb-f0c6-4533-8955-770b91faabf4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.362460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165512b-a4c0-44e4-9b04-f20b68adc7ad None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.408371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.408992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.409272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.409548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.410342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=35b4380116c842509fe369b502503cb8) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:33.410650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.413633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.419303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 180/355] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.421398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.471467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002eacb-f0c6-4533-8955-770b91faabf4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.479406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:33.522581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.523337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.523688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.524060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.527674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.527674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0747e0-19b9-43fb-a511-b6dff94ccea6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.529050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 176/356] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.530322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.530700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.531045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.557832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:33.559358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:33.569698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661252-1bd7-4579-aca6-180d888b1a1e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.617269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.618266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.618886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.619404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] PATH_INFO: `/v3/users/b778f138902a46b4aebd49ba85b4a454` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.620859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] RBAC: Authorizing `identity:get_user(user_id=b778f138902a46b4aebd49ba85b4a454)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.622660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.624603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 181/357] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/users/b778f138902a46b4aebd49ba85b4a454 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.625699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.640293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:33.661450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 177/358] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:33.663151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.776631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb7-7fcb-4316-80d3-d7200489633e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.785081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:33.803269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128355b-b5d3-4763-a8fb-2c3cc83c308e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:33.818420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.819164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.819521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.819875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.832218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:33.839474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:33.840408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:33.841226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 182/359] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:33.841889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.847916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:33.848787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:33.849806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:33.850262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:33.851229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=35b4380116c842509fe369b502503cb8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:33.851820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:33.853596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:33.859982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 178/360] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:33.861126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:33.926335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.004685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c87fd-1585-44e0-8587-8ffed84eb0e9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.051223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.051815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.052224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.053080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.062083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e0c0-3c51-4793-91bf-82fe0b61abde None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.068941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:34.075151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:34.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:34.076907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 179/361] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:34.077565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.151320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.151975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.152254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.152524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.153326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] RBAC: Adding query filter params (role.id=2aa3a46a15e347ceaebef17caa92ba7c, user.id=b778f138902a46b4aebd49ba85b4a454, scope.system=all) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:34.153652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:34.155074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:34.169659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 183/362] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 15:15:33 2022] GET /identity/v3/role_assignments?user.id=b778f138902a46b4aebd49ba85b4a454&scope.system=all&role.id=2aa3a46a15e347ceaebef17caa92ba7c => generated 510 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:34.172815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.220948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.236407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61ea-e77c-4a4b-9dab-500c358a7afd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.272334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.331404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e749-916e-49df-83a4-998bb5d9a378 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.352601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.353249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.353532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.353806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.354544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:34.354849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:34.356291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc7837-74d2-4437-8fd4-34f94213c0d5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:34.374790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.375775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.376296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.376954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.378089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:34.378591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:34.380214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50df-cc14-42fb-b8e0-9dc31c7d485c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:34.382227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 180/363] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:34.383511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a740d-8d0f-4eda-bb41-bbc866c58799 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.383798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a740d-8d0f-4eda-bb41-bbc866c58799 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.384095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a740d-8d0f-4eda-bb41-bbc866c58799 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.385305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 181/364] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:34.391046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 184/365] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:34.391603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a740d-8d0f-4eda-bb41-bbc866c58799 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.393524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.394020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.394493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.420382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:34.422302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:34.521654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 185/366] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:34.522968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.549501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a740d-8d0f-4eda-bb41-bbc866c58799 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.569997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.608446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.609456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.610808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.611292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.665767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2987f-27a2-468a-8b6b-9ccdd7bdcb4b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:34.675055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.707066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:34.709208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:34.710339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:34.711211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:34.711692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:34.712182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:34.793133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:34.795024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:34.830109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 182/367] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:34.831072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.872956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.905575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 186/368] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:34.906583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:34.916407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:34.952016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8bfb2-54ce-49e9-87e4-2fd376169733 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.007286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acc749-5959-411e-9631-a763840e880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.007286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acc749-5959-411e-9631-a763840e880e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.007843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acc749-5959-411e-9631-a763840e880e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.008233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acc749-5959-411e-9631-a763840e880e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.022574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acc749-5959-411e-9631-a763840e880e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a4acc749-5959-411e-9631-a763840e880e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:35.029863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:35.030888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:35.031886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:35.034241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 183/369] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:35.035128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acc749-5959-411e-9631-a763840e880e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.057545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1513b3-fce8-4ed8-8000-cc1ac096a10e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.080605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.104201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.105527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.105941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.106339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.107797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.109825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.112679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 187/370] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:34 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.113848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.163519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acc749-5959-411e-9631-a763840e880e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.212183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.215547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.215547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.215547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.216783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.218362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.220500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 184/371] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.227997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.233015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.266074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e806ff-f79a-4c72-a4d9-3b4227269936 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.312467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.316078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.316454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.316882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.317749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:35.318719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.320255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.333068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 188/372] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.334120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.372493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19a84b-d4ce-48ee-b9ca-c97d281c869e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.397905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.410848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.411734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.412116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.412505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.413831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.415473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.417549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 185/373] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.418415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.467381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095dc0d3-85e0-4ecb-9507-a69ffbfd61c7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.509279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.510375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.510667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.511047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.513285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.515052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.516192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.518072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 189/374] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.518946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.560070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f442a-9a33-442c-90a7-123db0328817 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.603574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.604449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.604982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.605366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.640118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048f6b-9827-4d69-b561-b156bcac4189 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.658506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.687886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.688428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.688892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.689415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.689834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.690366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2391ff9-ca8d-4f3c-a9b6-c3ff50623204 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.690936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:35.691394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.693275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.701768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 190/375] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.702836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.825513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:35.874604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 186/376] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:35.876123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b09695-1212-4bbd-b92c-821c0c2de22d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:35.876340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7a35-da3d-4903-b865-91093f319df2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.876738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7a35-da3d-4903-b865-91093f319df2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.877123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7a35-da3d-4903-b865-91093f319df2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.878850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 187/377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:35.883951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.884382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.884767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.905754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:35.908165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:35.916482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:35.917338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:35.917770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:35.918251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:35.919234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:35.919712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:35.921322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:35.928182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 191/378] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:35.929175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:35.988311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.050858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 188/379] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:36.078379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd755e7-f233-44fa-b154-c64d64ea244d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.093707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.148448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.149107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.149392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.149669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] PATH_INFO: `/v3/projects/c1de4e10bc0744aea22a689aefa058d1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.150864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] RBAC: Authorizing `identity:get_project(project_id=c1de4e10bc0744aea22a689aefa058d1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:36.152472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:36.154188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 192/380] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:35 2022] GET /identity/v3/projects/c1de4e10bc0744aea22a689aefa058d1 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:36.163853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.263931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc5283-c287-4b12-8b5c-7353f52de1b3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.265835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.297914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8e7e-b852-4067-9c54-d25615bf7627 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.314917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.322196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.323503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.324026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.324898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.340620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:36.345081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.345731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.346017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.346299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] PATH_INFO: `/v3/groups/4c75e167d54c4025840a78ef073fca9e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e064f2ac-75b3-44d8-9125-bca6c2c7eff1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:36.348208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:36.349156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:36.350455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:36.352944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 189/381] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:36.355005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcfacb-fd1d-48b9-9b12-8c6faf40494d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.355441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcfacb-fd1d-48b9-9b12-8c6faf40494d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.355881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcfacb-fd1d-48b9-9b12-8c6faf40494d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.356297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] RBAC: Authorizing `identity:get_group(group_id=4c75e167d54c4025840a78ef073fca9e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:36.357747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 190/382] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:36.362444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:36.362444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 193/383] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/groups/4c75e167d54c4025840a78ef073fca9e => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:36.364017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.364248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.364656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.365189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.392465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:36.394356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:36.487888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 191/384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:36.488995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.502966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2d4f1-2f7f-46d9-9d46-770e2ccb181d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.550110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.550839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.551230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.551616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.552540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:36.552959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:36.554522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:36.563657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 194/385] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:36.568431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.602129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.608757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.615235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc9515-a8fe-4e0e-a46f-e9352e297f49 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.657269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.658242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.658758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.659233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.660257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] RBAC: Adding query filter params (group.id=4c75e167d54c4025840a78ef073fca9e, role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=c1de4e10bc0744aea22a689aefa058d1) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:36.660790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:36.662507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:36.668713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 192/386] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/role_assignments?group.id=4c75e167d54c4025840a78ef073fca9e&scope.project.id=c1de4e10bc0744aea22a689aefa058d1&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:36.669775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.690907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234cc54-3fae-48af-a958-1221c667df5d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.736373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.737050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.737418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.737598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.749464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:36.758055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:36.759038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:36.759852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 195/387] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:36.769388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.822999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974089-7029-4692-bdac-fc60e4dd42c2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:36.895973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:36.896638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:36.898067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:36.898356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:36.900611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:36.902379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:36.904640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 193/388] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:36.910254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:36.940988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.958060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:36.966151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c13edf-c5a7-4e38-b591-e21f854c7e21 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.010744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.011377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.011658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.011995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.012771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:37.013079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.014493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.038112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 196/389] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:37.044512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:37.103634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb857d-ec3b-425f-9046-7308b4d45ac9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.150533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.151199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.151481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.151754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.155899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.157531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.159639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 194/390] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:36 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:37.182258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:37.244639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af542-e1c9-4b61-b029-c538ffeb80c3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.291491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.292203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.292490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.292772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.294673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.296188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.298612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 197/391] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:37 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:37.306478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:37.340147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:37.397793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:37.415730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26472d2a-8384-474d-b94c-ae6e4aaec7db None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.431155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a517f-a2c1-4bc5-baf9-4a7620c15b88 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.475918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.476566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.476850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.477129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.479575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.481051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.482875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 198/392] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:37 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:37.483355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.484050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.484334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.484672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.489534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:37.490581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, group_id=3b6c0b4b487a481a9a2ae68fd9f00d64, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.492043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2caf3c-13a5-4afe-b5f3-a4c4cf645883 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.519891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 195/393] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 15:15:37 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:37.628353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:37.632657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7c675-c3ea-4479-a3dc-06e79a8c31b7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:37.680553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:37.681186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:37.681469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:37.681752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:37.684064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b4380116c842509fe369b502503cb8, group_id=3b6c0b4b487a481a9a2ae68fd9f00d64, role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:37.685560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc545e-5cd7-4a76-b51a-7a29a0e0ce14 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:37.707352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 199/394] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 15:15:37 2022] PUT /identity/v3/projects/35b4380116c842509fe369b502503cb8/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:15:38.267321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d101676-3fa4-4f61-9231-1bd8edaa0242 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.267953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d101676-3fa4-4f61-9231-1bd8edaa0242 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.267953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d101676-3fa4-4f61-9231-1bd8edaa0242 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.269494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 196/395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:38.275151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.275447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.275731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.294812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:38.296131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:38.324791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fcc69-f374-4881-a713-00940c338c05 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.325454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fcc69-f374-4881-a713-00940c338c05 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.325945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fcc69-f374-4881-a713-00940c338c05 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.327787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 197/396] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:38.334130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.334727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.335228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.359361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:38.361069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:38.397636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 200/397] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:38.435307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:38.467707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 198/398] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:38.507659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:38.553437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:38.560722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3e5e-49e7-4d2d-a88c-e4c3d51d3265 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:38.621687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:38.622342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.622626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.622902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.654071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:38.655286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:38.667618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:38.668657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:38.669544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:38.675852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 201/399] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:38.680672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:38.699245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3cda6-4fbf-42a5-b2cf-f4339c491d33 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:38.775288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:38.776338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.776777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.777192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.800738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:38.806170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:38.807147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:38.808154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:38.819351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 199/400] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:38.823620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:38.890198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac424-9c0d-4628-a5c1-fa0710d24aed None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:38.948429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:38.949715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:38.950903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:38.951980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:38.953021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:38.954601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:38.957004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:38.976074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 202/401] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:38.978687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:38.988167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.042729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fd706-8bcf-4c56-8c50-3ac988a14801 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.100506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.101139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.101420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.101696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.102411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:39.102715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.107850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.117697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 200/402] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:39.123956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.128150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.162383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ce3f-c006-48b6-9786-dcd183430234 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.240717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.241376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.241654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.241924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.243196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.244671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.246273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 203/403] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:38 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:39.252653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.281533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b9618-8993-44f7-94f6-41445d97a14a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.326944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.327719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.328147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.328490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.329809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.331320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de4cb7-6783-4501-b95e-1913f473bd8b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.334097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 201/404] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:39 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:39.335490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827f82-d974-4918-acc0-485f0be355b7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.336080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827f82-d974-4918-acc0-485f0be355b7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.336593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827f82-d974-4918-acc0-485f0be355b7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.338554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 202/405] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:39.343535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827f82-d974-4918-acc0-485f0be355b7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.372076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.405519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca2b1d-4bfe-4f08-b3e0-30c0cba8b9d0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.442449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.456294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.457736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.458166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.458866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.460478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.462487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026a4ca-f766-4eaa-808a-eecb3d68c560 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.465289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 204/406] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:39 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:39.466728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.473045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.473045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.488944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827f82-d974-4918-acc0-485f0be355b7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.496362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:39.498299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:39.531405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.532075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.533162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.533489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.536859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.538533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.540548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 203/407] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:39 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:39.541294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.595212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 205/408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:39.596217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.663133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07fd54-9c00-47be-b0e6-8dc992298d06 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.694858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.706271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.707258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.707748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.708308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.711160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, group_id=3b6c0b4b487a481a9a2ae68fd9f00d64, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.712944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.724660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:39.730622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 204/409] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 15:15:39 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:39.734729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd3236-73bc-4f48-83bc-0f68c7663e1a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.736329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:39.778025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.778703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.778981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.779257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.786202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3c40360ddb43678c7d73ea54c5c576, group_id=3b6c0b4b487a481a9a2ae68fd9f00d64, role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:39.786202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:39.803453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 206/410] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 15:15:39 2022] PUT /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576/groups/3b6c0b4b487a481a9a2ae68fd9f00d64/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:39.939345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878a7e3-d719-4f5a-a7cf-07ba388e51a5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:39.985296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:39.985971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:39.986253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:39.986527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:39.999181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:40.004810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:40.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:40.007258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 205/411] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:40.015141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.159411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f01b50-44f8-48cc-ab1d-bf7d1e78c545 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:40.200347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:40.206645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:40.207273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.207557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.207856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.208626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:40.208933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:40.210321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e1587-396f-4735-bcb6-cb7abefa6b64 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:40.223748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 207/412] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:40.229827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.320000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:40.376710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48661ff0-354a-4a6b-80a7-fcbf06b246a9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:40.386142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e865d-9ac9-40ad-a4ca-33f42cf0bc60 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.386805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e865d-9ac9-40ad-a4ca-33f42cf0bc60 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.387303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e865d-9ac9-40ad-a4ca-33f42cf0bc60 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.389056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 208/413] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:40.401148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b37448-649c-4515-b5a4-c72d6b99269f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.401437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b37448-649c-4515-b5a4-c72d6b99269f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.401720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b37448-649c-4515-b5a4-c72d6b99269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.418090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:40.419028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.419507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.420015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] PATH_INFO: `/v3/groups/4c75e167d54c4025840a78ef073fca9e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.421419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] RBAC: Authorizing `identity:get_group(group_id=4c75e167d54c4025840a78ef073fca9e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:40.423071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b852-ab27-4f26-9c20-7627d6fb725c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:40.425478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 206/414] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/groups/4c75e167d54c4025840a78ef073fca9e => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:40.433971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9d24-07f5-4c03-88ce-28547ca8f744 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.433971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9d24-07f5-4c03-88ce-28547ca8f744 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.433971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d9d24-07f5-4c03-88ce-28547ca8f744 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.433971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 207/415] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:40.433971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d9d24-07f5-4c03-88ce-28547ca8f744 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.435395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-16b37448-649c-4515-b5a4-c72d6b99269f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:40.436795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-16b37448-649c-4515-b5a4-c72d6b99269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:40.508477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 209/416] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:40.510052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.510481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.510897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.528441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:40.529839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:40.578389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d9d24-07f5-4c03-88ce-28547ca8f744 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:40.606897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 210/417] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:40.608536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.612177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:40.617898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:40.619039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.619039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.619484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] PATH_INFO: `/v3/projects/c1de4e10bc0744aea22a689aefa058d1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.620792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] RBAC: Authorizing `identity:get_project(project_id=c1de4e10bc0744aea22a689aefa058d1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:40.622559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:40.624883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 208/418] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/projects/c1de4e10bc0744aea22a689aefa058d1 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:40.632093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.682853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:40.734068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1d3f2-85a9-4229-85ed-9ce27dc5d045 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:40.776164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371dbb-449c-4bb1-81f1-bee9543c732a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:40.779402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:40.780261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.780718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.781136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.795763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:40.802536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:40.803625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:40.804642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:40.807422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 211/419] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:40.808392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.821305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:40.822240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:40.822565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:40.822954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] PATH_INFO: `/v3/projects/c1de4e10bc0744aea22a689aefa058d1/groups/4c75e167d54c4025840a78ef073fca9e/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:40.825474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1de4e10bc0744aea22a689aefa058d1, group_id=4c75e167d54c4025840a78ef073fca9e, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:40.827038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:40.847856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 209/420] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 15:15:40 2022] PUT /identity/v3/projects/c1de4e10bc0744aea22a689aefa058d1/groups/4c75e167d54c4025840a78ef073fca9e/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:15:40.847856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:40.971692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.015011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aeacd-bf03-4f25-ae4b-b40d1f204bc0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.043876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.055869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761f6b-b929-4c53-9e56-9abe62ec5271 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.084963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.085190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.086488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.086488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.109325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:41.112822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:41.117178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:41.118099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:41.118892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:41.120088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.120600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.121088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.122033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:41.122535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:41.125224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 212/421] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:41.125629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:41.131545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:41.142457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 210/422] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:41.153060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:41.319974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc2fa8-9eff-46e1-8102-3bc372ae35bf None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.336829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3f95-168b-45bd-ab57-297fdd334e88 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.351944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.352563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.352843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.353116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.353935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:41.354242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:41.355651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:41.357938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.370804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 213/423] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:41.372410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:41.404538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.405329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.405964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.406079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.410160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.479969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982ffb-bdf5-4ba8-a11a-68284cb4950d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.516684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.517641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.518114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.518594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.599684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:41.601204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:41.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:41.614127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:41.640547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.719230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 214/424] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:41.725161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:41.815555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 211/425] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:41.821454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:41.846438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:41.941100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb098d-16e8-4298-b7f7-32391b419eb9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.994073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:41.994719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:41.995005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:41.995282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:41.997666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbe53-6fa0-4a20-8bc4-68ef92869880 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:41.999597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.001069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.002938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 212/426] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:41 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:42.010372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.073454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.074159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.074631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.074903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] PATH_INFO: `/v3/projects/35b4380116c842509fe369b502503cb8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.076239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] RBAC: Authorizing `identity:get_project(project_id=35b4380116c842509fe369b502503cb8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.077833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.080322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 215/427] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:41 2022] GET /identity/v3/projects/35b4380116c842509fe369b502503cb8 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:42.085946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.095620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.113292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.138196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e83ed-706c-48ef-be2f-47f283f5212f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.197622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.198217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.198470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.198749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.200383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.202154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75a845-436c-476f-9b1a-3af28342799c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.204429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 213/428] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:42.206013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b53-5e08-4cdf-a458-0d580a89f4a8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.206510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b53-5e08-4cdf-a458-0d580a89f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.207101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad3b53-5e08-4cdf-a458-0d580a89f4a8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.208587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 214/429] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:42.213373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad3b53-5e08-4cdf-a458-0d580a89f4a8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.246781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69096652-138b-4404-a9f5-1e9f6acee18f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.285076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.285820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.286100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.286396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.287526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.289132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc105e-931a-4548-afe7-ce550ccd5f55 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.295654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 216/430] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:42.297009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad795-9f23-43f6-98a0-22d0895a8edc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.297299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad795-9f23-43f6-98a0-22d0895a8edc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.297697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad795-9f23-43f6-98a0-22d0895a8edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.316521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8e4ad795-9f23-43f6-98a0-22d0895a8edc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:42.318433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8e4ad795-9f23-43f6-98a0-22d0895a8edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:42.321353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad3b53-5e08-4cdf-a458-0d580a89f4a8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.355668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.357040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.357325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.357600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.358388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=35b4380116c842509fe369b502503cb8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:42.358697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.360150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3089f-d853-4cf8-a44a-74bee2f1c2cc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 215/431] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87add4-e963-4302-81a9-f6cae50ee7fa None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87add4-e963-4302-81a9-f6cae50ee7fa None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87add4-e963-4302-81a9-f6cae50ee7fa None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 216/432] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:42.372130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87add4-e963-4302-81a9-f6cae50ee7fa None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.393591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.410555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 217/433] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:42.417346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.417633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.417949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.434994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.452545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:42.457749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:42.513371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87add4-e963-4302-81a9-f6cae50ee7fa None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.553894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.554876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.555140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 218/434] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:42.555629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.556048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.556462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.557442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=35b4380116c842509fe369b502503cb8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:42.557946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.559732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.566296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 217/435] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=35b4380116c842509fe369b502503cb8&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:42.583565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.671063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada319c6-d5dd-4272-9b5a-e7930c2a39cf None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.696892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.712375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:42.719922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.721173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.721608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.722026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.723001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080ddce-1e21-44d9-9d01-e2b9fadf2e1d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.742492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:42.748815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:42.749790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:42.750580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:42.751174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 219/436] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:42.756758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.768995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.769968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.770433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.770950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.783986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:42.790199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:42.791082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:42.791918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 218/437] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:42.799667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.875070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdb74c-ba61-473a-812a-82bd5063b555 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.922559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468f55-f272-4921-b864-a69305e14400 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:42.923486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.924407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.924732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.925003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.927924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:42.928245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.929730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.939589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 220/438] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:42.944929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:42.979259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:42.979907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:42.980190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:42.980463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:42.981181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:42.981482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:42.982858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:42.995902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 219/439] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:42.998757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.005063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.025144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.092228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e14a-e073-4a00-a84e-931418279238 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.122995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5ec9b-eb7d-42f9-95ab-d13229ace4a1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.133893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.134509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.134795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.135073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.163429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.164096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.164382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.164655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.219227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.220647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a1d85-6a9f-4daa-8b7e-15ea5df5f7cd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.246205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.247890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471c47e-e662-4e0b-907e-6b08f577d2b5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.252251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.279050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.326200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 221/440] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:43.327344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e462-df0f-4eb8-9550-095f01db0862 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.327633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e462-df0f-4eb8-9550-095f01db0862 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.327942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383e462-df0f-4eb8-9550-095f01db0862 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.329351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 222/441] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:43.331766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383e462-df0f-4eb8-9550-095f01db0862 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.374125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 220/442] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 15:15:43.374125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.374125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.374125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.389733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:43.390917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:43.451681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383e462-df0f-4eb8-9550-095f01db0862 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.460326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 221/443] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:43.467983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.496164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.496357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.496763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.497052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.497328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.498740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.500395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.502538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 223/444] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:43.503329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.530415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.565297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a4106-a0a6-47d8-adc9-378fec4f848b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.601990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.602942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.603229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.603498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] PATH_INFO: `/v3/projects/2a3c40360ddb43678c7d73ea54c5c576` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.604685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3c40360ddb43678c7d73ea54c5c576)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.606114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.607746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 222/445] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/projects/2a3c40360ddb43678c7d73ea54c5c576 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:43.608604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.615726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e254a-1c9e-4962-b16c-1dabd23db858 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.650639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.651269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.651544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.651823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.662113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:43.667052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:43.668043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:43.668857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 224/446] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:43.669513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.723656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea0ab8-3146-4bc2-94dd-ffad18953644 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.725206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.758004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.764337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.764978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.765255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.765525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.766665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.768126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.769704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 223/447] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:43.770466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.781337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0857-dfd1-4a40-a0ec-c47522bed8ed None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.814221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.814845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.815122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.815392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] PATH_INFO: `/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.816557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] RBAC: Authorizing `identity:get_group(group_id=3b6c0b4b487a481a9a2ae68fd9f00d64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.818617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.820359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 225/448] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/groups/3b6c0b4b487a481a9a2ae68fd9f00d64 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:43.821035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.871474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853f56-23d8-436a-bd57-93665e87dba2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.904723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.905327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.905603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.905873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.906567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:43.906867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.908259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.916751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 224/449] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:43.917266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40288e2a-8e54-4f8d-92c4-578dbe4a3724 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:43.917523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.951191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:43.951986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:43.952596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:43.952942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:43.953204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:43.953966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:43.954272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:43.955745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:43.962816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 226/450] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:43.962816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:43.984222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:44.035094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aaf91-cfdb-4159-acf5-0c8ce9ddddc9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:44.074752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:44.075381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:44.075661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:44.075956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:44.076793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] RBAC: Adding query filter params (group.id=3b6c0b4b487a481a9a2ae68fd9f00d64, role.id=63ebc483a8af4b2584cc1a6f03e1625e, scope.project.id=2a3c40360ddb43678c7d73ea54c5c576) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:44.077106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:44.078568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:44.084424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66300-dbde-4da1-979f-4cb85dec12bb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:44.092125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 225/451] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/role_assignments?group.id=3b6c0b4b487a481a9a2ae68fd9f00d64&scope.project.id=2a3c40360ddb43678c7d73ea54c5c576&role.id=63ebc483a8af4b2584cc1a6f03e1625e => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:44.137518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:44.138166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:44.138447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:44.138721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:44.213314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:44.214810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:44.230410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:44.316162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 227/452] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:44.321849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:44.417203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:44.423933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44f537-7c74-40fe-b833-79fb5dd9a027 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:44.457246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:44.457881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:44.458662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:44.458931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] PATH_INFO: `/v3/projects/c1de4e10bc0744aea22a689aefa058d1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:44.460127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] RBAC: Authorizing `identity:get_project(project_id=c1de4e10bc0744aea22a689aefa058d1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:44.461546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:44.463172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 226/453] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:15:44 2022] GET /identity/v3/projects/c1de4e10bc0744aea22a689aefa058d1 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:44.468758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:44.565894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4db4-f87f-4695-aa87-72339b146ce0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:44.579821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:44.600368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:44.601001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:44.601280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:44.601551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] PATH_INFO: `/v3/groups/4c75e167d54c4025840a78ef073fca9e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:44.602674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] RBAC: Authorizing `identity:get_group(group_id=4c75e167d54c4025840a78ef073fca9e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:44.604186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5a93b-6d72-4575-bde5-c2517c111085 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:44.605755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 228/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 15:15:44 2022] GET /identity/v3/groups/4c75e167d54c4025840a78ef073fca9e => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:44.611525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:44.707550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd85d-bfb8-4675-b09c-cca2bfa76db4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:44.739240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:44.739860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:44.740151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:44.740423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:44.741193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] RBAC: Adding query filter params (group.id=4c75e167d54c4025840a78ef073fca9e, role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=c1de4e10bc0744aea22a689aefa058d1) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:44.741502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:44.742902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b148b-5514-412d-8fc4-c71ee0091a55 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:44.748438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 227/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 15:15:44 2022] GET /identity/v3/role_assignments?group.id=4c75e167d54c4025840a78ef073fca9e&scope.project.id=c1de4e10bc0744aea22a689aefa058d1&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:47.052960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7745d-db28-443f-a81c-1746cb37157a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.053461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7745d-db28-443f-a81c-1746cb37157a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.053533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7745d-db28-443f-a81c-1746cb37157a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.055134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 229/456] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:47.060847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e543-6bc8-42ce-b6cb-286bbeac61d0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.061352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e543-6bc8-42ce-b6cb-286bbeac61d0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.061431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e543-6bc8-42ce-b6cb-286bbeac61d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.088007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-eb25e543-6bc8-42ce-b6cb-286bbeac61d0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:47.089219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eb25e543-6bc8-42ce-b6cb-286bbeac61d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:47.114086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:47.174440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 228/457] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:47.223110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7745d-db28-443f-a81c-1746cb37157a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.263987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc366-a029-44f1-9db7-99e0249d2554 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.264354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc366-a029-44f1-9db7-99e0249d2554 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.264552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc366-a029-44f1-9db7-99e0249d2554 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.266182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 229/458] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:47.272627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444dce2-94d3-49c5-97bc-09e0956c1df6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.273204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444dce2-94d3-49c5-97bc-09e0956c1df6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.273689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444dce2-94d3-49c5-97bc-09e0956c1df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.295395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9444dce2-94d3-49c5-97bc-09e0956c1df6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:47.296894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9444dce2-94d3-49c5-97bc-09e0956c1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:47.374355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7745d-db28-443f-a81c-1746cb37157a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.383510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 230/459] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:47.389398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22fb90-2344-4fc0-b951-ec83742c87b0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.389513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22fb90-2344-4fc0-b951-ec83742c87b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.389996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22fb90-2344-4fc0-b951-ec83742c87b0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.391203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 231/460] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:47.399397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.399686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.399990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.420485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:47.421116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.421396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.421665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.435619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:47.437376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:47.437964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:47.439167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:47.453933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:47.454998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:47.456050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:47.456050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:47.456050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.456050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:47.458557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 230/461] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:47.459565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.498897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:47.519998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 232/462] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:47.521805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.560834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:47.580091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baa786-c8b4-43ce-91c3-8d0d04383bbe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.617042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:47.617900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.618322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.618736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.630235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:47.631989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:47.647291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:47.648382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:47.649233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:47.649233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:47.649233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.649233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:47.650950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 231/463] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:47.651953 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.672357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e44a77-11b8-4085-8d3d-7afbbead6078 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.718955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:47.719651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.719984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.720258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.721452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:47.721626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:47.723640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:47.751982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 233/464] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:47.753202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.756908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:47.776427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e62d-329f-4aba-8902-13adfd869049 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.813049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:47.813863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.813987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.814230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.826839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:47.828349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:47.843573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:47.844466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:47.845306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:47.845306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:47.845306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:47.845306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:47.849622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 232/465] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:47.850814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:47.866340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:47.898684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd214df-b49e-4add-b5eb-48421e6db477 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.944226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:47.945294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:47.946224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:47.952778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:47.953701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:47.954687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:47.956523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:47.970598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e120ff-90c5-44d7-a4e0-0697e26aef0b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:47.974749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 234/466] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:47.976763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.005787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.006328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.006604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.006880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.007558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.009043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49615b-ce77-40ed-a0e0-565d37afa961 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.019296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:48.045016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 233/467] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:47 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:48.045016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f2092-0051-41c4-a711-87c737f15924 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.045016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f2092-0051-41c4-a711-87c737f15924 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.045016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f2092-0051-41c4-a711-87c737f15924 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.045016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 234/468] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:48.046060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f2092-0051-41c4-a711-87c737f15924 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.100374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498a332-7480-4f2b-b9ac-90c199473262 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.140721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.141501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.142069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.142349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.143059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:48.143523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.145912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc061692-9a1f-4b7c-b717-73be860e8ad5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.154652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:48.161924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 235/469] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:48.163792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4881-dea5-4682-963a-4067389166ec None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.164200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4881-dea5-4682-963a-4067389166ec None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.164685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4881-dea5-4682-963a-4067389166ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.184966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f2092-0051-41c4-a711-87c737f15924 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.189760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-af1f4881-dea5-4682-963a-4067389166ec None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:48.191094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-af1f4881-dea5-4682-963a-4067389166ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:48.236002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.236788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.237074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.238158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.238858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.240863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.267111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 235/470] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:48 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:48.267273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.278393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 236/471] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:48.312232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f4881-dea5-4682-963a-4067389166ec None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.341484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:48.427977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064d9b-6915-46e1-a7a1-3adf9c234bc8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.469845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f4881-dea5-4682-963a-4067389166ec None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.501446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.502065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.502347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.502716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.503397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.505053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.507858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:48.524249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.524868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.525165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.525419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.537151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 236/472] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:48 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:48.544074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.546004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:48.564365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:48.565280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:48.566142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:48.566142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:48.566142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:48.566142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:48.566142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 237/473] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:48.571672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.696241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa339a-444b-472b-9c88-979d5dabd4ea None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.707182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:48.738970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.740476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.740954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.741432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.742339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:48.742795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.744201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe016-ae82-4004-a856-41fa7d783360 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.757448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 237/474] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:48.762645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:48.893152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad0ff-78de-46f4-8e80-9525dcf3c2c9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:48.893498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79fa6-0915-4b91-b98c-acf1fc9e0faf None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.893833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79fa6-0915-4b91-b98c-acf1fc9e0faf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.894111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79fa6-0915-4b91-b98c-acf1fc9e0faf None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.896043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 238/475] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:48.902543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.902898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.903237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.938398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:48.939619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:48.943334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:48.943993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:48.944330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:48.944609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:48.945299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:48.946735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:48.984558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 238/476] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:48 2022] POST /identity/v3/users => generated 249 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:15:49.048571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 239/477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:49.084223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:49.126754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:49.199647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e3f-7b90-4180-a37b-8c4e629d8fc0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:49.235156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:49.235773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:49.236087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:49.236365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:49.237086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:49.238567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:49.249774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 15:15:49.249774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 15:15:49.249774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [parameters: {'id': '4863dec1f3974bd788f6bcf9437c8b5c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 15:15:49.249774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:15:49.267916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:15:49.268801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 15:15:49.269711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 15:15:49.271148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:15:49.272080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:15:49.272885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'id': '4863dec1f3974bd788f6bcf9437c8b5c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:49.273664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 15:15:49.274543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:49.275386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 239/478] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 15:15:49 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:49.276764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:49.377926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:49.384693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac2400-1cab-44de-97f8-3abc2646093b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:49.426179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:49.426850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:49.427129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:49.427400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:49.439409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8748a2d8-ea26-475f-80a9-7fb183f893f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:49.445000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:49.445907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:49.447359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 240/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:15:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:49.452186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:49.568549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:49.572304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738664-eebd-41b2-b79b-5cc3f9e74770 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:49.620098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:49.620720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:49.621001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:49.621471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:49.622204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:49.622507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:49.627910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdfce1-ab7c-434d-b4c6-81a4c0d4e879 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:49.637068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 240/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:50.547318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f69ca-7853-4b53-910d-05bfa0ecf11d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:50.547854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f69ca-7853-4b53-910d-05bfa0ecf11d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:50.547934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f69ca-7853-4b53-910d-05bfa0ecf11d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:50.549505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 241/481] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:50.554938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:50.555357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:50.555544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:50.577311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:50.578501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:50.678049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 241/482] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:50.712291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f69ca-7853-4b53-910d-05bfa0ecf11d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:50.846623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f69ca-7853-4b53-910d-05bfa0ecf11d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:50.892546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:50.893551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:50.894027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:50.894499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:50.912682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-07e51b87-d0a4-49d4-9f1d-dc0cd6044dc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:50.917937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:50.918792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:50.919575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:50.922177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 242/483] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:50.928203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.054662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.063238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca435f9d-4448-4099-803b-39ccc59eeaab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.088263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c54f6-70e3-4707-9580-9cd21e865497 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.088857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c54f6-70e3-4707-9580-9cd21e865497 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.089351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c54f6-70e3-4707-9580-9cd21e865497 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.091148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 243/484] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:51.095891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149497-6616-46c0-bdca-ebab388110ff None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.096392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149497-6616-46c0-bdca-ebab388110ff None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.096914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149497-6616-46c0-bdca-ebab388110ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.103864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.103864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.103864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.103864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.104206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:51.104426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.105705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d78a-31b7-48db-8ce6-2713c80a9597 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.114851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 242/485] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:51.116209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ae7b-67bc-4f57-b355-748bb880de6b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.116503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ae7b-67bc-4f57-b355-748bb880de6b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.116778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ae7b-67bc-4f57-b355-748bb880de6b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.118252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 243/486] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:51.121598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4ae7b-67bc-4f57-b355-748bb880de6b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.128132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-48149497-6616-46c0-bdca-ebab388110ff None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:51.129576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-48149497-6616-46c0-bdca-ebab388110ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:51.141995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.224240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 244/487] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:51.226856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.228436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.231642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.253905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4ae7b-67bc-4f57-b355-748bb880de6b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.256406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:51.257825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:51.291096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.291747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.292047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.292325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.304227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.305786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:51.322469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:51.323379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:51.324765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:51.324765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:51.324765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:51.324765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:51.324765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 244/488] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:51.325197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.336526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.337358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 245/489] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:51.338603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.440306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.466687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca4769-d548-4bb5-b653-35ca979879fa None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.479344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5b751-07df-452d-9df9-a02dfd44e5c2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.501397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.502484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.502484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.502841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.503709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:51.504143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.506136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.518053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 246/490] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:51.518434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.519148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.519431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.519985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.521014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.534735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:51.543194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:51.544102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:51.544896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:51.552135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 245/491] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:51.553120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.622132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.624058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869179a-0a24-4555-a754-93bc783711d7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.660220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.661096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.661469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.661865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.663578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6928b-3929-4ab8-8bc9-972c0d59744c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.667365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.674471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:51.679664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:51.680606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:51.681408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 247/492] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:51.682069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.710582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.711148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.711430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.711703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.719601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.721119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6543e2bb-e17b-41ef-8b4b-6806705097ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:51.738205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:51.740303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:15:51.741433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:51.743871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 246/493] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:51.745392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c90-a8a3-4bdf-b55d-92393a85c888 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.745685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c90-a8a3-4bdf-b55d-92393a85c888 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.745970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c90-a8a3-4bdf-b55d-92393a85c888 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.747254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 247/494] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:51.747977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c90-a8a3-4bdf-b55d-92393a85c888 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.806145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd922ce-f6eb-428a-9187-e44a495b555e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.841590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.842202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.842479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.842752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.843535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:51.843762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.845166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.857980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 248/495] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:51.857980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:51.866202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c90-a8a3-4bdf-b55d-92393a85c888 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:51.893867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.904650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:51.905276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.905561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.905842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.906567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:51.906868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:51.908292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1d5e1-dda0-4e3e-9854-a4e2ede453cd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:51.916863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 248/496] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:51.918772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:51.922543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:51.922871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:51.923155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:51.942620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:51.943818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:51.970525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b786c43-c864-4467-8e62-887c95d88a45 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.006530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.007153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.007439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.007715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.008548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:52.008856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.010318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.023912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 249/497] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:52.023912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.050983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 249/498] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:15:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:52.052793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.129986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eaf79-7317-4227-a4b0-e694183ad05f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.149249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.168211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047472e-8181-45b8-9eab-37d26553aec7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.172032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.172930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.173444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.173917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.185894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.187340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.189010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.206948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:52.207223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:52.208083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:52.213853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:52.213853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:52.213853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.213853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:52.214088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.214088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.214088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.214280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 250/499] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:52.214280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.224480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.225909 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:52.241730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:52.242634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:52.243482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:52.243482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:52.243482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.243482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:52.243699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 250/500] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:52.244493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.315706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10a44-2ef0-489e-9534-a8bdf50a1d0c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.343432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17416acd-5ded-4bed-8f1a-41cc9bdffaf3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.348732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.349366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.349646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.349915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.359938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:52.365875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:52.366929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 251/501] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:52.367734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.383449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.384531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.385018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.385515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.401129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.424625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.457742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.459221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.470101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117ba7-c529-4fcb-9e10-458e5d2d0626 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.505495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.506041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.506317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.506587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.507269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:52.507572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.508954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff3084-bca8-4021-954d-cf9764037ad1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.521118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 252/502] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:52.522258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055288-c745-4295-a0cd-62115b85a7b7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.522550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055288-c745-4295-a0cd-62115b85a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.522823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055288-c745-4295-a0cd-62115b85a7b7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.528613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 253/503] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:52.528613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91055288-c745-4295-a0cd-62115b85a7b7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.559080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 251/504] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:52.560075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.615069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.625463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91055288-c745-4295-a0cd-62115b85a7b7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.626503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.659181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.659789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.660107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.660271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da5282-94a4-42d1-b683-71b7c7ded4b7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.660387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.661691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:52.661998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.663479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.674749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 254/505] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:52.675377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.694203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.694819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.695100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.695368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.696113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:52.696420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.697796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d200b-b159-4bc2-96fb-ba148a2f3c6e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.706824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 252/506] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:52.708582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.708875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.709165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.726660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:52.727882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:52.784090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eace3-8e66-4120-aede-d7048904ac65 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.792199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 253/507] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:52.793008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.818290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.818933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.819213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.819486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.827370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.828832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:52.840569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:52.841514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:52.842402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 255/508] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:52.843553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.857908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.863020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:52.903656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a2ef5-7dd0-4bd5-86a5-d1e2ff36d38f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.936650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.937286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.937566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.937841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.945957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6409d-f349-4843-8228-2bf28a06bd17 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:52.949758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.949885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:52.965682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:52.966672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 254/509] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:52.967741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:52.982736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:52.983491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:52.983769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:52.984072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:52.991475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:52.992933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:53.004931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:53.005843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:53.006732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 256/510] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:53.007279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.072106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165d823-f229-4929-a603-ab95b04ec10a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.077674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.085885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.107142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.107774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.108087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.108365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.110707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323225c-3f7f-40fe-b4ed-194223d36248 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.118713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.120215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:53.134601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:53.135414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 255/511] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:53.140965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.143478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.144135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.144414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.144683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.156129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.157549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:53.171388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:53.172239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:53.173081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:53.173081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:53.173081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:53.173081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:53.173081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 257/512] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:53.173336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.240228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39979d9-7d14-4ffe-89fe-2f849a207737 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.273837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.274396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ab3f-8d95-4dec-a831-8ec8ededbc10 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.274498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.274717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.274989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.275785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:53.276116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.277560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.293999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 256/513] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:53.294796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.296629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.314987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.315609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.315927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.316201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.317496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:53.317800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.319163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.325034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.331423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 258/514] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:53.332252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.403274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71cf1-fd81-4d60-8e88-b4f1ed027952 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.430719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6ff6e-4c30-4bce-9d8e-e757ba0a904f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.436394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.437012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.437295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.437569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.438385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:53.438690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.440144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.457878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 257/515] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:53.458676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.469912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.470520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.470798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.471069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.471751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:53.472083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.473459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.484357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 259/516] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:53.485123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.528396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.538395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.558065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce71eb-e027-44f9-9d1f-320529b73d21 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.585834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4036-30b7-4574-a882-bba576e61b24 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.592062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.592673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.592958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.593234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.593924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:53.594229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.595617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.606746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 258/517] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:53.607534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.623319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.623976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.624265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.624544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.691074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.692710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.707336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f170-c047-4eb9-a105-ce2ec3d44269 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.734440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.740959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.741617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.741903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.742184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.748722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.750187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.752640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:53.762975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:53.763813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:53.764672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 259/518] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:53.765177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.794590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 260/519] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:53.795431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.864634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0ae2c-317f-4e10-8cef-558c789e5c05 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.893964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154da67-2951-4dff-80d7-05cc50d4559f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:53.896608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.897224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.897506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.897780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.927894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:53.928527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:53.928814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:53.929087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:53.936203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.937658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:53.950379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:53.951274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:53.952201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 261/520] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:53.952743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:53.954954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.972049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:53.974662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:53.976647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.052388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dfd20-391b-46b7-8314-5075b8a44430 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.073591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 260/521] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:54.074689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.085695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.086310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.086597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.086875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.087578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.089054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.109949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 262/522] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:15:53 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:15:54.110744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.176199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.198672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.202813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b372d7-f403-4718-9d23-03c40f5f612d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.228080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e34b-aaed-4e32-839b-5f88d18365e7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.244694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.245442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.245724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.245984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.246754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:54.247057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.248560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.267270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 261/523] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:54.267958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.272360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.273032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.273317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.273600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.284987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.287552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:54.303508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:54.304656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 263/524] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:54.305484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.380289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93d683-4f42-4ef2-bc95-72673a737c2b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.418589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.418821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070d4d-078a-432c-9ef7-5dafbefecf9b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.419208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.419491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.419766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.420878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:54.422011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.423482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.429894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.443943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 262/525] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:54.449452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.456745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.457451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.458245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.458639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.469383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.470591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.470822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:54.485228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:54.486104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:54.487063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:54.487063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:54.487063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:54.487063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:54.487063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 264/526] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:54.487360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.563743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49be7-87f6-4e84-9d23-f7ae070827e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.595370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6586-fd34-4186-aa50-8cf185a0097b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.599725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.600439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.600790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.601138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.601999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:54.602380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.604005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.619859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 263/527] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:54.619859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.646911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.647567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.647868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.648153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.648854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:54.649159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.650556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.664048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 265/528] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:54.664988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.699538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.740077 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.767672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d054f-5b52-4fe3-9434-4722c322ca42 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.805942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e866-7eed-481e-8d3f-b5271859bd51 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.823150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.823916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.824551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.824890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.844506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:54.845416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:54.845874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:54.846321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:54.847209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:54.847697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.849377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.861503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 266/529] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:54.862605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:54.906544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:54.908260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:54.952421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:54.973106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c7f7a-7f37-46fc-a228-030220f02adc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:54.996856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.009529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93840-0188-4674-9424-445ed8451528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.010487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93840-0188-4674-9424-445ed8451528 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.010967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93840-0188-4674-9424-445ed8451528 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.011438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93840-0188-4674-9424-445ed8451528 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.021390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 264/530] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:55.021390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.024228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93840-0188-4674-9424-445ed8451528 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.025884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93840-0188-4674-9424-445ed8451528 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-31e93840-0188-4674-9424-445ed8451528 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:55.038664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:55.039517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.040387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:55.047116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 267/531] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:55.048198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93840-0188-4674-9424-445ed8451528 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.125238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3ad7-2f41-431c-8661-c8ed7b5f6a47 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.154904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93840-0188-4674-9424-445ed8451528 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.161607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.162246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.162528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.162865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.170113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.171890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:55.184384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:55.185306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.186132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:55.186547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 265/532] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:55.187774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.196016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.196891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.197348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.197791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.209651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.211333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.213382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:55.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:55.228721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:55.229567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:55.229567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:55.229567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:55.229567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:55.235893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 268/533] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:55.236967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.240260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.301093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab909d3-181a-467e-aa7e-7c4cf5174854 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.342275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.343017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.343309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.343590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.344406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:55.344699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.346153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.346925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78904896-f5cd-410a-a7f3-70e797def94a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.364033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 266/534] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:55.365000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.383857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.385272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.386085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.386641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.388013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:55.388795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.390523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.410633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 269/535] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:55.411781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.471743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.478645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026c32-5c77-4f76-ab1a-65ce4b259912 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.480470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.517044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.517709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.517995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.518334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.518951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:55.519256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.521193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.521330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffc48c-76aa-4e9f-8b0f-5ae767ca81ae None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.538174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 267/536] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:55.538174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.567979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.569085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.569722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.570079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.571232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=77f96ae3547b4be6bd6bb31c14903926) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:55.571619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.573887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.587931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 270/537] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/role_assignments?user.id=77f96ae3547b4be6bd6bb31c14903926&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:55.590610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.708743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05f50-e976-43ab-b48a-7b5297316ca4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.758705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878db1e-e0fa-4134-bc1f-33add4dd0d30 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:55.770539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.772408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.772877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.773300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.774287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=711efe02d15146dfb6a1a737abcee3a1) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:55.774748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.776405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.787970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 268/538] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/role_assignments?user.id=711efe02d15146dfb6a1a737abcee3a1&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:55.794820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.814264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:55.815313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:55.815598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:55.815901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:55.824466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:55.825961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:55.839933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:55.840847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:55.842161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 271/539] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:55.847484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:55.880452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:55.996345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9135553-67d0-4545-9fea-82d16d0e30db None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:56.032201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:56.033055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.033482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.033903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.034798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:56.035261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:56.036882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f78c-5647-46f5-8b3d-8d9525a1c37c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:56.051118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 269/540] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:56.057116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:56.189711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da9eba-e765-4a5e-b172-a0b3a30717a9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:56.228901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:56.229756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.230180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.230594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.231524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=94eb8ac7bd934c318f0e52833928c589) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:56.232020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:56.233581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916cef5-dff0-4f52-aabe-a5fec9c495b3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:56.239946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 272/541] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:15:56 2022] GET /identity/v3/role_assignments?user.id=94eb8ac7bd934c318f0e52833928c589&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:56.327521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3576-73a3-4ca8-a86a-5344a2dd77e5 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.327950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3576-73a3-4ca8-a86a-5344a2dd77e5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.328239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3576-73a3-4ca8-a86a-5344a2dd77e5 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.329843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 270/542] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:56.346014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.346634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.347109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.373593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:56.375159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:56.429048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:56.474224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 273/543] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:56.504705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c3576-73a3-4ca8-a86a-5344a2dd77e5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:56.612314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:56.633548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c3576-73a3-4ca8-a86a-5344a2dd77e5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:56.674913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:56.675557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.675853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.676699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.691653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:56.696849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:56.697722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:56.699195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 271/544] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:56.705984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:56.823363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcd22-4dea-4bfe-b5b2-71220af952c9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:56.875345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:56.875918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.876208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.876488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.877209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:56.877522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:56.878896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4474-e44f-465c-b09c-396549e0a4b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:56.887876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 274/545] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:56.893453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:56.969886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7583d-c38b-4138-97c2-5b3f98516187 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.970181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7583d-c38b-4138-97c2-5b3f98516187 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.970465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7583d-c38b-4138-97c2-5b3f98516187 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:56.972069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 275/546] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:56.977096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:56.977389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:56.977671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.005769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:57.006976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:57.065595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d7a7-7742-48be-a1c1-38f3d3ec5824 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.077382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:57.117958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.118608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.118884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.119155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.137352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:57.138772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:57.157779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:57.171364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:57.172294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:57.173944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:57.173944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:57.173944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:57.173944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:57.173944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 272/547] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:57.176488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 276/548] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 15:15:57.178019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.201565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.318250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ce4a-5887-4d67-8d5c-cda15accaa90 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.356943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.357961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.358246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.358522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.370273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:57.375402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:57.377604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:57.380147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 277/549] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:15:57.388298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.412257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:57.426293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0cead-1910-46eb-9c2f-33bb6e3731ac None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.493463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa66f4-7da3-4238-be5d-6c68cc3dadd6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.501453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.502284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.502715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.503145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.504523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:57.504975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:57.506528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:57.531854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 273/550] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:15:57.532053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.532053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.532053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.532053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.532614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:57.533375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.534000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:57.535942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:57.556318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 278/551] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:57.560763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.651867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:57.704224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:57.712397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fd1b-07cb-4346-ac56-91ff611fa3b9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.741961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567038-d300-4aab-aadd-e8c44fc300c3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:57.751260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.751937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.753079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.753395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.777358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:57.778817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:57.803877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:57.804844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:57.805795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:57.805795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:57.805795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:57.805795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:57.810697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 279/552] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:57.812444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:57.813252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:57.813597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:57.813903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:57.815307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.835305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:57.836889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:57.852473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:57.854674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:57.856312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 274/553] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:57.859693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:57.993340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75328-e2c2-4e45-8d9d-9da9d4e8b5db None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.028818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.029578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.029959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.030328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.031232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:58.031633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.033255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.044206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.047173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 280/554] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:58.053324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.078650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc57b9a-49cd-4421-a8b5-31b041ad062c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.137423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.166184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1fb49-b195-4115-9904-ab015966b2e5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.177502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.178130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.178686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.178971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.179787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:58.180230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.181695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3396-e0d3-4161-9fd1-b5acef302064 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.201182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 275/555] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:15:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:58.207035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36b2e-1f26-45a7-9855-0259f17618e4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.207523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36b2e-1f26-45a7-9855-0259f17618e4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.208018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac36b2e-1f26-45a7-9855-0259f17618e4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.208389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.209053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.209319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.209587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.209686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 276/556] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:58.210470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac36b2e-1f26-45a7-9855-0259f17618e4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.227466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.229782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fc06b856-1bb0-43fb-b4f1-ce636c4c1127 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:15:58.245897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:58.246863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:15:58.248247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:58.249730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 281/557] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:58.251155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.252240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.252240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.271058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:58.272302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:58.329847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac36b2e-1f26-45a7-9855-0259f17618e4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.330477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.346589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 282/558] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:58.347427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.373561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.374178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.374463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.374736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.421278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.445864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.447340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbecd39-19d1-4be9-8b29-d2e51f2899e5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.460215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d781c4-8872-47df-a777-3442c3bb194c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.497848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.498392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.498673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.498944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.499735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:58.500068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.501481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05da0d-4f54-474f-b632-e0a5d42d4ca9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.539875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 283/559] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:58.540636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caf7af-ca9a-4159-ad45-fa5cbccbff97 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.541020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caf7af-ca9a-4159-ad45-fa5cbccbff97 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.541200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caf7af-ca9a-4159-ad45-fa5cbccbff97 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.542391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 284/560] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:58.543078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4caf7af-ca9a-4159-ad45-fa5cbccbff97 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.550772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 277/561] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:15:58.552673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b88b4f-4044-4928-b8be-1ab7b47bfbd7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.552964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b88b4f-4044-4928-b8be-1ab7b47bfbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.553240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b88b4f-4044-4928-b8be-1ab7b47bfbd7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.554476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 278/562] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:15:58.555239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b88b4f-4044-4928-b8be-1ab7b47bfbd7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.599561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.630894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.653085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4caf7af-ca9a-4159-ad45-fa5cbccbff97 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.654844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b88b4f-4044-4928-b8be-1ab7b47bfbd7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.686111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.686750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.687038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.687317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.687622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.688285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.688566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.688839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.697830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:58.699568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.701197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fb24185e-5456-4c73-b97a-185dd1fba9dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:58.703111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:58.704578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:58.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:58.709235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 285/563] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:58.710392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5839f7-f697-4c9e-86cf-c33ea478570f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.710684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5839f7-f697-4c9e-86cf-c33ea478570f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.710962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5839f7-f697-4c9e-86cf-c33ea478570f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:58.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:58.717185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:58.718092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:58.718092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:58.718092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:58.718092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:58.718092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 279/564] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:58.719060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.728619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca5839f7-f697-4c9e-86cf-c33ea478570f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:58.730100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca5839f7-f697-4c9e-86cf-c33ea478570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:58.795811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 286/565] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:58.797237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.797530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.797804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.817794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:15:58.818961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:15:58.822361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24f5ff-3144-4e6f-88a6-9103c5146989 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.846674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.848515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:58.857272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:58.857902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:58.858181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:58.858452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:58.869312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:58.870781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:58.884815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:58.885894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:58.887109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:58.887109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:58.887109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:58.887109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:58.887374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 287/566] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:15:58.887374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.888525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 280/567] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:58.889501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:58.991504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9600a-feaa-4d9a-9e58-04be67a322ab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:58.992716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301a4f-96b1-43fe-8048-eeaf8889e423 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.025488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.025972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.026106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.026385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.026498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.026659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.026761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.026940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.027365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.027620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.027706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.027950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.029059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.029314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.037501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 288/568] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.038209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.041887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 281/569] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.044747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.066961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.086418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.138377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91c235-a87d-4e74-af02-6fd32cca696a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.144374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ed1e-9943-404a-90b7-5489cf824562 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.171577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.172246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.172528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.172802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.177338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.177945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.178240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.178502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.179193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.179500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.180908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.182783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:59.188156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.189103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:59.189957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 289/570] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.190633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.196765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 282/571] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.197706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.277089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.290197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9ac44-5918-45fe-b547-385a6857037f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.294014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.296957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9949c-309d-49d3-a06b-f3a63e054da1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.323415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.324266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.324546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.324821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.330032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.330656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.330937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.331214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.336027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:15:59.341925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:15:59.342236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.343167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:15:59.344052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:59.345455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.345546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 290/572] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.345546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:59.361693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.362611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.363487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:59.363487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:59.363487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:59.363487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:15:59.363487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 283/573] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.364038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.450299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edb735-e5d7-473e-9869-c3555debf017 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.465011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9861-557f-446c-9e95-19cea971c0f6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.483818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.484480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.484759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.485030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.492747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.494191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.505940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.506556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.506841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:59.507034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.508077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:59.509021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:59.509443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.509443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.509577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 291/574] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.509651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.509719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.509807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.521990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 284/575] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.521990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.522680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.538165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.609585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e9599-196b-4d8e-9dcc-40a74b1b8d1a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.622030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36854f28-d235-4a5f-a917-97695adb4963 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.642561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.643191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.643571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.643737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.651080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.652551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.654763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.655371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.655645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.655945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.656650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.656951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.658288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:15:59.664446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.665473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 292/576] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.666547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.669429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 285/577] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.674388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.731299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.751432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.765217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38eca-2e76-4bc8-9133-9db4e695b3a2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.774714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fe071-5b82-4cea-9cf7-c7b0ca12823a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.797704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.798314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.798596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.798868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.799546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.799869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.801266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.806513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.807111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.807389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.807660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.808430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.808730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.810143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.812638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 293/578] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.813417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.823511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 286/579] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.824609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.912346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f8cf2-0647-4832-888a-920fa0f9bdf7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.923203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908647ed-18fe-4b91-9539-e2fd0d81cc45 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:15:59.937857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.946604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.947256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.947543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.947822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.958251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.959690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.965054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:15:59.965189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:15:59.965414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:15:59.965683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:15:59.966442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:15:59.966743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:15:59.968202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:15:59.970691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:15:59.975479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:15:59.976354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:15:59.977338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:15:59.977338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:15:59.977338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:15:59.977338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:15:59.977338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 294/580] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:15:59.977618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:15:59.984616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 287/581] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:15:59.985386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.075404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53064b5-508c-4dd8-92a3-b1fc867ba241 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.086215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f8c6d-6396-4171-8de4-7330562be900 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.108179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.108810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.109092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.109363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.118477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.119108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.119394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.119669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.121013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.122454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.127033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.128501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:00.136406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:00.137468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:00.138379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:00.138379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:00.138379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:00.138379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:00.138379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 295/582] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:00.138660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:00.141711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:00.142678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 288/583] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:15:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:00.143613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.173366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.202395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.242303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534d958-e799-433a-993d-d3d0887af9c0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.245648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9cb9-fb1c-49e1-801e-7815c0699db0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.275835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.276615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.276897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.277174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.278847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.279445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.279722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.280027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.280717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:00.281018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.282379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.291208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=fe58a99358bc42aebeb126e5ef2cf21a, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.292646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.294146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 289/584] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:00.294926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.307970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 296/585] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:00 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:00.308787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.398562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.414193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145aa0c6-72d7-4d32-81b7-7c4620f51c87 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.420293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dad82-47b8-4474-b6b8-62006ecb72db None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.436413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.459938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.460632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.460916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.461195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.462000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=eeec63e358424f57ae02d5693b359805) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:00.462306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.463667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.463790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.464575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.465029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.465443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.466281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:00.466594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.467991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.470323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 290/586] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/role_assignments?user.id=eeec63e358424f57ae02d5693b359805&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:00.470942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.482605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 297/587] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:00.483505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.610462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39618e09-1dde-4b2e-bec8-e691140b6812 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.610854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245b58d-37a6-4009-b6a6-12f8ab04bc9c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.651034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.651656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.651764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.651985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.652242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.652458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.652740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.653012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.653114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:00.653357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.654815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.661921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.663366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.673895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 291/588] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:00.674759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:00.675493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:00.676485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:00.677773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:00.682080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 298/589] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:00.682270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.685167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.726780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.794138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226411-d83e-4d88-a59b-38ad635540cf None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.804033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaef0b-6bc9-4209-a2d3-96b8297e3667 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:00.832113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.832761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.833043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.833317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.840925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.841832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:00.842409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.843035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:00.843459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:00.843889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:00.844824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:00.845286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:00.846830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:00.854801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:00.855971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:16:00.857024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:00.860613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 292/590] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:00.861465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.873965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 299/591] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:00.875256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:00.941274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.970925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:00.977268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707316e8-ba72-4096-9854-c257bca0bae9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.000150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bc457-c9fd-4b8b-8376-26525f2e0b2f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.021966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.023484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.023891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.024271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.041734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.043239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.049325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.049946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.050225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.050501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.051367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.051674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.053174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:01.062464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.063367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 293/592] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.069071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.079200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 300/593] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 15:16:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:01.080081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.190878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b86668-75c8-41ff-bf20-a367cf691cfd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.199561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852e4aa-3134-4758-affe-3d2aa116ccb6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.211147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.229428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.229972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.230253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.230525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.231208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.231509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.232888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.240605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.241500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.242041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.242510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.249140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 301/594] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:01.249140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.261234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.262818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:01.280778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.281786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.282979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:01.282979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:01.282979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.282979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:01.287380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 294/595] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.288426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.292439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.369426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdafed-826a-485c-a83b-a2b59a2ccadd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.394624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861eba-0d8d-4b68-a091-586083acf9f7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.407411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.408380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.408863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.409354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.421011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.422717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.430253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.430672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.430952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.431227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.439069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:01.440121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.440997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.442432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:01.442432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:01.442432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:01.442432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:01.442676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.444914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 302/596] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.446155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:01.454853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.455818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.456818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:01.460829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 295/597] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.461683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.484007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.526811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.568928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123244e-adc3-467a-915a-900957dadd68 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.573987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e7076-d825-4a1d-bf6b-a86ce451681f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.613924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.614542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.614819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.614988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.615114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.615574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.615770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.615882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.616116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.616217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.616813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.617118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.617490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.618473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.630786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 296/598] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:01.631621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.639361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 303/599] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:01.640370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.755355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11304425-4956-4f54-a612-72b3584e1882 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.761019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b6c6-e790-465e-a99f-01b4a998999a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.784261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.789737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:01.791978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.792754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.793148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.793543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.802561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.804201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.804659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.805432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.805870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.806299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.807211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.807663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.809241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:01.816653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.817685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:01.818746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 304/600] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.819341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.830028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 297/601] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:01.831111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.928763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf8228-0e48-477e-9b30-b339d02fb687 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.933566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2a1a9-9a1e-445b-93ba-144880bc8023 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:01.971378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.971979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:01.972110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.972321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.973130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.973700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:01.974124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:01.974538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:01.975435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:01.975921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.977532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.980449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:01.981786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:01.996035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:01.996959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 305/603] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:01.997840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:01.998887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 298/603] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:02.000900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.009691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.046772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.106702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451082-3aab-4b37-8cd3-4322f1e05ed0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.108143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d447e17-b2fb-4a46-8ec8-e340615d47e4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.144145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:02.145756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.146160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:02.146468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.146829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.147108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.147381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/77f96ae3547b4be6bd6bb31c14903926/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.147562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/94eb8ac7bd934c318f0e52833928c589/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.162970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=94eb8ac7bd934c318f0e52833928c589, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:02.163355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=77f96ae3547b4be6bd6bb31c14903926, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:02.164822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:02.165265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf7b96-b9fa-4546-836f-01b74058c6ca None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:02.186948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 306/605] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:01 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/77f96ae3547b4be6bd6bb31c14903926/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:02.187465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 299/605] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:01 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/94eb8ac7bd934c318f0e52833928c589/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:02.188378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.292299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.337406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c6b0d-3a0b-4301-bd3c-841955ef4edd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.378290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:02.379105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.379371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.379645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.380444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:02.380748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:02.382190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:02.396131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 307/606] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:02.399969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29239a-3997-43a0-a2e8-de8367c97a84 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.400927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29239a-3997-43a0-a2e8-de8367c97a84 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.401576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29239a-3997-43a0-a2e8-de8367c97a84 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.403239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 300/607] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:02.407354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.409014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.409302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.409624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.436284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:02.438511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:02.530730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 301/608] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:02.553696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67989fc9-8637-4db0-89b0-2cea48d5f90c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.571177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.586967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.613672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:02.614363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.614645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.614921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/711efe02d15146dfb6a1a737abcee3a1/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.616397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.637969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=711efe02d15146dfb6a1a737abcee3a1, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:02.640127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621b829-997f-4ff3-9427-dcfa86fccf9a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:02.659257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 308/609] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:02 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/711efe02d15146dfb6a1a737abcee3a1/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:02.661773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f4b-c1eb-4c68-a671-8406c2b4f779 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.662259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f4b-c1eb-4c68-a671-8406c2b4f779 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.662649 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f4b-c1eb-4c68-a671-8406c2b4f779 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.668366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 309/610] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:02.670588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.671100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.671596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.690690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:02.692949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:02.717328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fc0ed-0375-446f-bfdb-05456378715f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.765602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:02.766283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:02.766567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:02.766845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:02.784003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 310/611] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:02.795358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:02.801072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:02.802061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:02.802901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:02.810261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 302/612] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:02.811703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.861451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:02.955457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:02.959966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb930b-fadd-4072-afb5-74e7ef44629a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:02.977647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:03.001369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.002315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.002794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.003261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.004736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:03.005302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.006861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.017070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 311/613] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:03.023633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.026756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c001b-70f4-4126-9cdf-821b1c9246c6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.093428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.093999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.094280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.094552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.120398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:03.127228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:03.128216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:03.129131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:03.138966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 303/614] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:03.141549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.191748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a28f53-7506-4f97-8d71-ec5d278a80df None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.228455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.229437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.229960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.230522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.242211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.243969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:03.259482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:03.260385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:03.261275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:03.261275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:03.261275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:03.261275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:03.267771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 312/615] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:03.269205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.299714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:03.306288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23363-9765-41bc-8c25-cad04eabea60 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.342913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.343581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.343811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.344165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.344887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:03.345192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.346597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.355163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 304/616] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:03.361223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.365709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:03.461392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec22eb-f5a8-4d26-9201-838620a8a969 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.528543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.529614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.530360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.531596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.532815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:03.534034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.538197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb77f-1057-4e2f-a3f5-f4a256ef6456 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.541131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.569190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 313/617] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:03.571693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.615226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.616346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.616826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.617298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.638017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.639800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:03.665711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:03.666660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:03.667573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:03.667573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:03.667573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:03.667573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:03.670491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 305/618] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:03.678610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.720491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:03.766491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b06e-498a-4975-9475-c847be72e167 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.789054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:03.836633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.837264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.837540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.837807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.847889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.850333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.861277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ef4b-6a7b-4d77-891e-9a6b545789b4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:03.865899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:03.866810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:03.867918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 314/619] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:03.872395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.929859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:03.930481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:03.930759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:03.931034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:03.931727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:03.932294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:03.933423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:03.958687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 306/620] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:03.964875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:03.991385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee22bf-f6cd-4f4e-8b7a-58977c79e2b7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.026951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.027947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.028445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.028893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.029849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:04.030342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.032059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.040159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.052812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 315/621] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:04.065548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.113890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422742a7-f17a-49ab-a489-c13e74bef34e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.131923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.154276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.154922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.155198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.155467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.169168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.170633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:04.195523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:04.196561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:04.197504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 307/622] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:04.206557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.248011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a20b-59ef-4b91-92d0-178b99328801 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.321530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.322158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.322448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.322724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.354008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7c869-1d6e-4549-b98c-0367cf8434c1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.411441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.412377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.412940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.413383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.414365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:04.414840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.416584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.441099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.447878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 308/623] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:04.449647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.474500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.476123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2b559-a544-46e2-8c14-95f4b3fcd1f3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.487311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.596596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05144b74-1ad0-428a-8026-64abfa3b8f3b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.652564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 316/624] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:04.653866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e543e-5169-4358-b559-18b0b267f50f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.654155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e543e-5169-4358-b559-18b0b267f50f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.654437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e543e-5169-4358-b559-18b0b267f50f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.655689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 317/625] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:04.658890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e543e-5169-4358-b559-18b0b267f50f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.666170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.666758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.667037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.667311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.679512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.681129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-ed23df66-a9fa-4fa5-91c6-997fec03435f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:04.698293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:04.699512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:04.700433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:04.700433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:04.700433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:04.700433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:04.700433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 309/626] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:04.703855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.703855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.703855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.723282 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:04.724501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:04.780582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.793583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e543e-5169-4358-b559-18b0b267f50f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.795498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 310/627] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:04.796528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.826041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:04.833991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.834635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.834911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.835390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.845864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.847012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4d14fbd7-d7fe-429a-afba-789e262562c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:04.863512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:04.864429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:04.865439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:04.865439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:04.865439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:04.865439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:04.865439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 318/628] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:04.866591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166747-a35c-4c75-b3eb-80a41796678f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.866591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166747-a35c-4c75-b3eb-80a41796678f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.866591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166747-a35c-4c75-b3eb-80a41796678f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.867731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 319/629] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:04.869849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73166747-a35c-4c75-b3eb-80a41796678f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.918813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ddb09-c301-4a22-802a-992fa154ab0b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:04.958542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:04.959540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:04.959649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:04.960053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:04.963311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:04.963311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:04.963311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:04.980017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 311/630] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:04.981019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:04.984401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73166747-a35c-4c75-b3eb-80a41796678f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.018535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.019198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.019494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.019771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.030320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a8162491-0f44-470e-95dd-d0d94d9f9721 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:05.036347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.037261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:05.038108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 320/631] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.038813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.039203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.039300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.046482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.058512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:05.060043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:05.070132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.093538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48bb0-179d-46e2-b115-afbd54aa31e2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.126214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 321/632] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:05.127044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.127599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.128249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.128528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.128802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.129489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:05.129793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.131161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.143240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 312/633] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:05.143240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.233856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2033bb-b604-49d1-88aa-f9cb25cf423e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.245857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1468-eacf-45ea-b08a-a1811f39dd89 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.266792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.267421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.267700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.268002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.270517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.275941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.277895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.280200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.280813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.281094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.281367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.282081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:05.282417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.283784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.286059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fa687843-5d6a-43c6-b09d-9d21652d452c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:05.289805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.290903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:05.292029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 322/634] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.292632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9f24-4bc3-4324-83bb-624eb031c508 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.292632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9f24-4bc3-4324-83bb-624eb031c508 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.292632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9f24-4bc3-4324-83bb-624eb031c508 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.295861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 323/635] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:05.295861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9f24-4bc3-4324-83bb-624eb031c508 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.304216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 313/636] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:05.304216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.396281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9f24-4bc3-4324-83bb-624eb031c508 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.398458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752666-2af2-4672-9e20-efeae1a72e5b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.428839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.429462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.429745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.430016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.430976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.431603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.431919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.432193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.436664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.438094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.442685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:05.448122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.449381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:05.450244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 314/637] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-df41ddb3-c0c4-4d3e-a544-96330f8dd989 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:05.450925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.451710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:05.452692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:05.453108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.453187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 324/638] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.453187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.453187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.453387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.485351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:05.486465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:05.512577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.536326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.554716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 325/639] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:05.555539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.558928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eb5c0-4b8e-4953-b5d8-3d4c071c6908 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.593809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.594420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.594698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.594976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.595717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:05.596049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.598013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.609404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 315/640] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:05.610173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.654697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c1e47-93cb-44fe-8c6e-7f2a219746f5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.687425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.688105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.688376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.688653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.699392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.701086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.706991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e82e42-5bbd-455a-8aa6-5186a42e711a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:05.715659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.716660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 326/641] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.717601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.727496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.745404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.745923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.746202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.746496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.747196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:05.747499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.748922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.754814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.756990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 316/642] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:05.757779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.821373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913b732-a5f3-4f01-a4bc-e7f0e9c827d4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.854103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25d6c8-55af-4700-b215-04a7082ae86a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:05.854990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.855602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.855916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.856190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.856929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:05.857232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:05.858637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:05.870739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 327/643] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:05.871498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.889449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:05.890086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:05.890367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:05.890643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:05.900753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:05.906111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:05.907063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 317/644] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:05.908449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:05.952865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.963493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:05.970025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e32d-12f5-4fdd-989a-b2a33d01be7e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.004461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.005065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.005347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.005627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/eeec63e358424f57ae02d5693b359805/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.010945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271b602-25dc-44a8-bf10-addc29a4217d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.019661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=eeec63e358424f57ae02d5693b359805, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.021123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.033087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 328/645] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:05 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/eeec63e358424f57ae02d5693b359805/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:06.033868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.046581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.047556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.048039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.048323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.049047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.049355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.050897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.064751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 318/646] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.065550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.153739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9c9b-3107-4ac1-b71e-f2fcb5a584b8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.175232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.178654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870c6fe-8b9f-45aa-ad33-e8e02c216fe4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.204667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.205316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.205597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.205875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.216363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.217813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.219181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.219940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.220262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.220541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.221367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.221676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.223159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.225206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.230733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 319/647] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.231555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:06.236236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:06.237125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:06.238551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:06.238551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:06.238551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:06.238551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:06.240516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 329/648] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:06.241577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.366376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667f498-9fe0-4960-97c2-b2ec81015e67 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.379002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b800de4-be04-48a5-9d41-17a31c470fb6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.409188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.410665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.411160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.411553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.412419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.412823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.414302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.423744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.424349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 320/649] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.424845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.425271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.425474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.425862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.436354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.438840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:06.453267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:06.454427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:06.455479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 330/650] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:06.456164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.467899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.542090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.561807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b57f99-5d80-4b9b-8f93-91db9f41ac23 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.587873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7cde5-ccd9-449e-b175-0b7d660798fc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.600592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.601989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.602472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.602998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.603934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.604445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.606229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.619939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 321/651] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.622770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.623413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.623685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.623985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.625012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.635802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.637264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:06.652746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:06.653870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:06.655064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:06.655064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:06.655064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:06.655064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:06.655064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 331/652] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:06.655434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.721051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.738347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108710d-3378-4541-95f6-0c0f4b9e7148 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.771432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1f2e-c096-4416-8b1c-962863ce6fe5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.776599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.777350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.777732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.778109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.779112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.779527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.781226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.782715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:06.808014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 322/653] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.809436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.810217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.810865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.811145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.811417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.821135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.822585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:06.836012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:06.837028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 15:16:06.837962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:06.841933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 332/654] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:06.842827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.925139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca872db-ecdf-4271-ad80-e2a3546c554e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.946002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ec933-e5a4-4deb-a7c6-25c8c743221c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:06.963546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.964639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.965163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.965638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.966535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:06.967221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:06.968840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:06.981786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:06.982407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:06.982683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:06.982952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:06.983968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 323/655] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:06.984996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:06.996998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.036815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.070808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.072485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.131200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452ecb-5db7-4db0-9819-fae3d23967e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.169483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.170258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.170642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.171074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.172030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:07.172477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.174013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.199163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 324/656] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:07.199163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.208965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 333/657] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:07.209963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.277640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.291144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.324728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662de163-a4eb-4be1-8013-04712e726930 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.331547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286beef-01f6-46e7-bef6-175463b79163 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.360567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.361271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.361466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.361734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.378159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.380061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.380541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.381197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.390808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.393233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:07.408996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:07.410498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:07.411431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:07.417955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 325/658] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:07.417955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.447406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.450289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.538903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5133-49e7-4302-a47a-b41571448d9b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.546943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.567441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.586918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.588196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.588881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.589326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.595898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 334/659] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:07.595898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.608569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.611575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:07.631976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:07.633085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:07.634168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:07.634168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:07.634168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:07.634168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:07.635952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 326/660] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:07.636932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.701953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df96d-533c-4ef3-b01f-ac14c0bd00bd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.743600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.744435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.744822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.745205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.746432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:07.746432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.748026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.771533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 335/661] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:07.771533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.786419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14bc4-a3bc-4d44-b159-1bff04cb10c9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.802032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.835782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.837571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.838131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.838581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.852426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.854581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:07.877632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:07.879250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:07.880164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:07.880164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:07.880164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:07.880164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:07.885621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 327/662] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:07.885621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:07.900126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e92b-85a2-430f-ab34-5a3c2c5458b2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:07.915035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:07.950581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:07.952194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:07.952657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:07.953103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:07.954540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:07.955155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:07.956855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:07.972726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 336/663] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:07.973819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.006315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3a6f0-d1eb-4d7f-aed2-204a3358cf46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.044020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.044911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.045289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.045681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.087796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.102903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121703-433c-4021-9266-2d6b8dc5852e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.120109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.121577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee18b2-904c-4898-98e8-ca535019a6c5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.139041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.139663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.140004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.140279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.140964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:08.141268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.142609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.150218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.155183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 337/664] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:08.156695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.241527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 328/665] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:08.242714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e36671-64d9-46c5-a524-d8aeff40e266 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.243003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e36671-64d9-46c5-a524-d8aeff40e266 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.243284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e36671-64d9-46c5-a524-d8aeff40e266 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.244734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 329/666] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:08.249203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e36671-64d9-46c5-a524-d8aeff40e266 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.270125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272a50-033b-4029-b8a0-b8004714546d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.306542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.307547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.307846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.308132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.314833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.316359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:08.328189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:08.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:08.329992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 338/667] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:08.330998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.342977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.365634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e36671-64d9-46c5-a524-d8aeff40e266 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.393156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.404102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.404771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.405050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.405322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.412793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.414236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f79716d4-eac2-451a-8ed6-7d12e9015917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:08.425450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:08.426489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:08.427566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 330/668] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:08.428120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621fa5a-ee2a-464f-bd43-6ce8599576d9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.428120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621fa5a-ee2a-464f-bd43-6ce8599576d9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.428654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621fa5a-ee2a-464f-bd43-6ce8599576d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.438027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a64-0567-485b-bdc9-e34a4033da19 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.445283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a621fa5a-ee2a-464f-bd43-6ce8599576d9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:08.446470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a621fa5a-ee2a-464f-bd43-6ce8599576d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:08.475150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.476245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.476731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.477227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.490066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.491522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:08.506490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:08.507474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:08.508726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:08.508726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:08.508726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:08.508726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:08.512202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 339/669] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:08.513025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.516366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 331/670] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:08.517674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5174d-96c6-4cbf-bf3a-576d84b64db8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.517965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5174d-96c6-4cbf-bf3a-576d84b64db8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.518255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5174d-96c6-4cbf-bf3a-576d84b64db8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.524307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 332/671] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:08.524307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5174d-96c6-4cbf-bf3a-576d84b64db8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.575537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.617189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eba1c-3eb0-46db-b09b-728ca69d7066 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.623869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5174d-96c6-4cbf-bf3a-576d84b64db8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.639748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.650956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.651576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.651893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.652172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.652927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:08.653237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.654662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.659382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.660631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.660915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.661182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.661927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:08.662231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.663655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d15bbe-ddf1-463b-a855-e846e979791d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.667028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 340/672] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:08.668003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.677037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 333/673] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:08.678243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.678531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.678813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.696714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:08.697898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:08.764445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 334/674] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:08.765194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.765828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a5119-e650-4c0f-9ce5-bf6c38e3f708 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.790566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.807420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.808059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.808342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.808610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.809291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:08.809597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.810942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.821854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 341/675] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:08.822630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.873207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a3b6-4c5f-4f8e-b1ab-4b99cecba5ee None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.880173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:08.907351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.908012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.908295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.908567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.918930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:08.923992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:08.925184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 335/676] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:08.927218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:08.931119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a3fb8-5daa-4d8f-bd0b-1f581be16bdf None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:08.967576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:08.968314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:08.968665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:08.968949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:08.969739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=77f96ae3547b4be6bd6bb31c14903926) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:08.970052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:08.971485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:08.977852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 342/677] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/role_assignments?user.id=77f96ae3547b4be6bd6bb31c14903926&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:08.978628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.031339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be056efe-53e5-4cb9-ac67-4d1bdc0d537c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.037454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.069707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.070428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.070806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.071119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.072045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=94eb8ac7bd934c318f0e52833928c589) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.072421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.074237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 336/678] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/role_assignments?user.id=94eb8ac7bd934c318f0e52833928c589&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.088947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.104331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f2182-bb35-4095-bf82-4cdfc04bd361 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.145586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.152163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.152895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.153184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.153462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.172288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:09.179244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:09.180264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:09.181102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:09.181760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 343/679] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:09.183191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.241837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a5e9d-15be-4583-83c1-c41ba89ee417 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.283164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.283739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.284048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.284380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.294039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.295574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:09.310448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:09.311559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:09.312548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:09.313440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 337/680] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:09.313718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391710e-9c1c-4d5d-907a-9cf8eac310bc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.314253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.317193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.353538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.354204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.354490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.354770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.355492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.355882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.357318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.366752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 344/681] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.367650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.435470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.446907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bfce3-9153-4db3-8689-39293ac8718e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.484208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.484923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.485278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.485628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.486418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.486797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.488981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.498134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 338/682] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.499150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.502759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d15f4-102c-45d3-a489-3e88c412a5e3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.551754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.552335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.552618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.552893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.553647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.554421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.556306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.576285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 345/683] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.577832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.608783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.631486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec05a0d-ead0-4cee-9eee-c5114edf34ef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.666785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.667430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.667713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.668025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.678921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.680431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.686369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a384-5336-427e-adb2-797148479d3e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:09.696004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:09.698002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:09.700389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:09.700389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:09.700389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:09.700389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:09.703537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 339/684] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:09.703537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.707396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.724825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.725475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.725757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.726031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.737438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.738930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:09.755533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:09.757639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 346/685] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:09.759845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.810225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d03f-a336-4fc7-bce4-0c76319ba43c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.844420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.847982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.849524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.849960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.850384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.851454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=711efe02d15146dfb6a1a737abcee3a1) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.852171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.853791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.860968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 340/686] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/role_assignments?user.id=711efe02d15146dfb6a1a737abcee3a1&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.861982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:09.875019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ab947-fac7-4606-a186-8a8d37d00dc2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.940391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:09.945472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:09.946110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:09.946399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:09.946671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:09.947379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:09.947688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:09.949371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:09.978279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155296a-37a8-4b25-a2c2-8d9d07c5be54 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:09.984486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 347/687] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:09.985820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.021712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.022641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.023074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.023509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.024447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:10.024991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.026691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.041303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 341/688] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:10.053017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.163291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a00cc-c3b8-4bc9-bd69-825d5119d69c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.200951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.201699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.201980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.202350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.208658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd8-d77b-4d31-a158-9391d54c984c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.210802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.212387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.214197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:10.227860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:10.228893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:10.229888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 348/689] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:10.236883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.241436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:10.251314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.251876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.252164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.252437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.260956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.262435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:10.275397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:10.276524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:10.277455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:10.281992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 342/690] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:10.283363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.386582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa751c-18b5-4a6c-bfad-0f3ce230c7a9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.402813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11e54f-f612-4a06-a5bf-9262beb6ded2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.435507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.436725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.437383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.438001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.439011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.439279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:10.439626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.439956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.440160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.440267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.441551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:10.441861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.442945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.443301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.468115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 343/691] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:10.476785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.481741 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 349/692] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:16:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:10.485056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:10.565437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:10.594978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:10.661989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b451c-8518-442d-b993-297b0b5d9ad7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.677229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7ba3d-ab40-488c-8906-a9ff3c47d509 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:10.724602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.725140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.725420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.725693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.740511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:10.741139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:10.741420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:10.741693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:10.850767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.852272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.875164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:10.876669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:10.942044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:10.984075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.045423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 344/693] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:11.052308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.066938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 350/694] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 15:16:11.073907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.191646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f548-76be-4acc-b2fa-5ac70ced03f7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.232716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.233358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.233644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.233919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.241408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13166-ee93-445f-8349-c729f56fc7e3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.244580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.246232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.254475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-673aae20-7f3b-4838-9dc3-f04e0dd39179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:11.264258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:11.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:11.266241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:11.266241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:11.266241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.266241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:11.266241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 345/695] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:11.266876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba19a8-ba3d-4a8c-80ea-0f34988bd58b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.267169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba19a8-ba3d-4a8c-80ea-0f34988bd58b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.267452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba19a8-ba3d-4a8c-80ea-0f34988bd58b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.272792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 346/696] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:11.272792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba19a8-ba3d-4a8c-80ea-0f34988bd58b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.297337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.297988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.298272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.298549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.314266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.316050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b77412e0-4d81-4311-9be1-f84bd29c34d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:11.336562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:11.337521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:11.338460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:11.338460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:11.338460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.338460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:11.340154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 351/697] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:11.341581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610d01-def0-41e6-a340-a3716de223ab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.342062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610d01-def0-41e6-a340-a3716de223ab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.342545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610d01-def0-41e6-a340-a3716de223ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.366314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5b610d01-def0-41e6-a340-a3716de223ab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:11.368516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5b610d01-def0-41e6-a340-a3716de223ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:11.389441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.390987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba19a8-ba3d-4a8c-80ea-0f34988bd58b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.428065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.428682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.428970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.429244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.429940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:11.430247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.431636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.447475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 347/698] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:11.447475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.453269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 352/699] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:11.459879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9df45f-afff-4d86-9ade-b47bb920b20a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.459879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9df45f-afff-4d86-9ade-b47bb920b20a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.459879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9df45f-afff-4d86-9ade-b47bb920b20a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.460150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 353/700] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:11.460697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9df45f-afff-4d86-9ade-b47bb920b20a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.509558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.561795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada6bef-ba4c-4947-99fd-e81663e56e42 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.585843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9df45f-afff-4d86-9ade-b47bb920b20a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.602649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.603400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.603698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.604129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.604986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:11.605625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.607358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3c37-a12e-4eac-8387-efab21aac174 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.626819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 348/701] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:11.626819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.626819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.626819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.630131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.630784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.631069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.631370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.641244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.643051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.644648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:11.645243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.645843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:11.658300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:11.659227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:11.660693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 354/702] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:11.661260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.721446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 349/703] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:11.723392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.772652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.792994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b891e9-7fa6-4923-b96b-9fa075410b0d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.828042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70cbb8-e61c-4eb1-be14-05734f41a549 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:11.831115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.831787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.832097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.832373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.843413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.844887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:11.859919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:11.860837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:11.861739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:11.861739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:11.861739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:11.861739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:11.861739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 355/704] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:11.862027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.872123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:11.873189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:11.873471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:11.873799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:11.881620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:11.883091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:11.894692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:11.895585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:11.896532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 350/705] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:11.897025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:11.926746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:11.984968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329cb0-a23a-4018-8de0-e15c724a3500 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.010703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e66c5-b347-4760-990b-da213f24b554 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.020093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.022620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.023633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.023970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.024317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.025302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.025685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.027518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.041927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 356/706] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.043573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.057008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.057853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.058242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.058640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] PATH_INFO: `/v3/services/network` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.059420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.061135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:12.074173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:12.075451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:12.076530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:12.080195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 351/707] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:11 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:12.081035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.157277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c472486-e88d-4808-bd53-be8073ffccbb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.169689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.197979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.198885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.199165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.199439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.200182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.200492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.201892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.220215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 357/708] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.220215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.305976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aec25-4926-4237-b6ca-e205136fdb81 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.329017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51f68-361c-418d-bb29-b02fca4afe20 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.340281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.340901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.341182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.341462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.342214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.342523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.343994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.356484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 352/709] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.357172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.370217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.370839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.371121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.371397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.372240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.372553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.373965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.380330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 358/710] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.381167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.401199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.427151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.495325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4002fa4-f912-4f4f-9dfc-0047d87acb35 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.520495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b251de-cd3e-4e2f-b6ef-00d446bd3d63 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.534915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.535541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.535850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.536119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.536800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] RBAC: Adding query filter params (name=network) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.538212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.538503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.543924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 353/711] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.543924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.563356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.563934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.564219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.564492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.566120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.567807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38bbda-1d1c-47c5-b45c-21c5899ecd12 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.594083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 359/712] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:16:12 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:12.595332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464b61f-beba-4991-b36d-b9a6e8a1c00a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.595623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464b61f-beba-4991-b36d-b9a6e8a1c00a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.595926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464b61f-beba-4991-b36d-b9a6e8a1c00a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.597415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 360/713] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:12.602474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464b61f-beba-4991-b36d-b9a6e8a1c00a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:12.681696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.692342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e189-ad0d-4bb0-819e-ba00f763600c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.720526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:12.735395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464b61f-beba-4991-b36d-b9a6e8a1c00a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:12.747310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.748911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.749394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.750076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.751552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=eeec63e358424f57ae02d5693b359805) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.752335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.755180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04115c6-7ec4-413f-a030-21d936884c64 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.763113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 354/714] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/role_assignments?user.id=eeec63e358424f57ae02d5693b359805&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.765192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.765925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.766382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.772597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:12.773244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:12.773527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:12.773804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:12.774603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] RBAC: Adding query filter params (type=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:12.774918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:12.776726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:12.788424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 361/715] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:12.790034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:12.791592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:12.877353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 355/716] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:12.899579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:13.028258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b62a-701a-4f0c-a74f-4ca8518d0f9d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:13.064067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:13.064730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:13.065014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:13.065296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:13.066000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:13.067373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:13.079138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:13.080137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:13.080967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 362/717] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 15:16:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:13.090238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:13.206919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:13.212628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5307712-6ae2-4afd-a745-fde8c474faa9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:13.248957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:13.249588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:13.249871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:13.250146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:13.250884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:13.251195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:13.252607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a385c-ef81-4bf9-8260-73c616cc12ab None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:13.258183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 356/718] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:13.263550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:13.297873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:13.372438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580526b-f2b9-4ab6-9445-ef1325b03c11 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:13.408683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:13.409338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:13.409628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:13.409910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:13.410591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:13.410900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:13.412315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49182f-36b1-478a-be60-e87fa34a612a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:13.417358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 363/719] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:14.644650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5738297-af13-4d15-8765-f20466416c3d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:14.645181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5738297-af13-4d15-8765-f20466416c3d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:14.645181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5738297-af13-4d15-8765-f20466416c3d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:14.646752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 357/720] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:14.652157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:14.652573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:14.652763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:14.675181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:14.676398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:14.746865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 364/721] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:14.773152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5738297-af13-4d15-8765-f20466416c3d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:14.928642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5738297-af13-4d15-8765-f20466416c3d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:14.996723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:14.997722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:14.998208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:14.998709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.016777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b00c033f-566b-478a-8d00-b0dd766e7c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:15.022613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:15.023570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:15.024483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:15.026785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 358/722] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:15.032643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.064233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.198850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8235eb-5622-405c-a9fa-e7a942289848 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.245050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4585d1-a8ad-43a6-a86d-c7e6342109c4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.245359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4585d1-a8ad-43a6-a86d-c7e6342109c4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.245631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4585d1-a8ad-43a6-a86d-c7e6342109c4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.247234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 359/723] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:15.248002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.249009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.249505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.249989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.250925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:15.251429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:15.253094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:15.259695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e95c7-02c4-44fd-88a2-6075e42d35f4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.260049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e95c7-02c4-44fd-88a2-6075e42d35f4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.260325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e95c7-02c4-44fd-88a2-6075e42d35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.263577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 365/724] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:15.270414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.299257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fb7e95c7-02c4-44fd-88a2-6075e42d35f4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:15.302595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fb7e95c7-02c4-44fd-88a2-6075e42d35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:15.326791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.375021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.394364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 360/725] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:15.396058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07691-b41b-46cd-93cc-01939eecdce7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.396544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07691-b41b-46cd-93cc-01939eecdce7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.397052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07691-b41b-46cd-93cc-01939eecdce7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.402534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 361/726] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:15.403059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b707-4b23-4e90-bd89-c937a4418251 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.404116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.404617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.405102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.424410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:15.426529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:15.440717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.441365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.441647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.441925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.453357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:15.454721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:15.470037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:15.470960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:15.472422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:15.472422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:15.472422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:15.472422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:15.472422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 366/727] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:15.477291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.500927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 362/728] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:15.509045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.587462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009383c2-3b38-4b28-9a40-78faad26b019 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.604176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.615345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.624739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.625370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.625653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.625929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.634853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fd54c-4647-48c1-9baa-d2dffdd8c0c6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.640260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:15.646191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:15.647157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:15.648066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 367/729] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:15.648778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.675803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.676596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.676875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.677148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.677834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:15.678153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:15.679503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:15.696214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 363/730] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:15.696214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.764178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2300-33bd-48dc-b070-9ee33648cc52 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.800787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.801513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.801727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022741b2-0dc9-4e8f-b937-0a38f76fca5d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.801834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.802069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.802761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:15.804160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90701b4-64da-44b0-94e0-0bfcf12ee81d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 368/731] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:15 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48884-b678-4bfc-8f10-b85b4a11b292 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48884-b678-4bfc-8f10-b85b4a11b292 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48884-b678-4bfc-8f10-b85b4a11b292 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 369/732] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:15.827871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c48884-b678-4bfc-8f10-b85b4a11b292 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:15.850874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.858287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:15.859023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.859309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.859585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.860350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:15.860651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:15.862333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e519-f614-47fa-b46e-9ec8db0bb32d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:15.872193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 364/733] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:15.873511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:15.873798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:15.874073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:15.888393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:15.894884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:15.896141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:15.965995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c48884-b678-4bfc-8f10-b85b4a11b292 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:15.979853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 365/734] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:15 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:15.979853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.006038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.006703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.006974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.007249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.016134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.017507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-bc4c5e46-3b92-4d48-b25e-99914ef2a4df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:16.032077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:16.033077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:16.034034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 370/735] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:16.034729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64dbb-191f-416c-8ead-d277df5190af None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.034729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64dbb-191f-416c-8ead-d277df5190af None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.034964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d64dbb-191f-416c-8ead-d277df5190af None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.036331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 371/736] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:16.036878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d64dbb-191f-416c-8ead-d277df5190af None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.100936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ff648-5478-4001-8b52-d8271c189b42 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.126988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.142917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.143580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.143888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.144166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.149124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d64dbb-191f-416c-8ead-d277df5190af None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.154112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.155569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.158160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-51422253-a998-4987-b991-5854fc6d9bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:16.172390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:16.173323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:16.175682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:16.175682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:16.175682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:16.175682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:16.175682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 366/737] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:16.176003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.176291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.176523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.192970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.193644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.198061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.198061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.200588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:16.200588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:16.212100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:16.219333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:16.221054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 372/738] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:16.228557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.277377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 367/739] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:16 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 15:16:16.279444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.343772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575773e7-5007-480d-b8db-97f9efb6af4a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.380549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.381123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.381408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.381687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.382480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:16.382790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.384272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.393182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867d8a7-e5a1-47cf-ac17-0758bb52ba4f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.396787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.407116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 373/740] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:16.408084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.430705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.431333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.431614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.431945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.432644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:16.432948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.434018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.434350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.451675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 368/741] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:16.451675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.519122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce556d1-440b-4110-9dec-b2781d18bff5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.556611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.557168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.557439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.557540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0b25-26e0-498f-906c-189ba553c53a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.557716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.558451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:16.558743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.560137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.568675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 374/742] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:16.569473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.600121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.600749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.601034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.601313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.602009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.603370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.620891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 369/743] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:16 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:16.622058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.664170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.682083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.700441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be906f-06e1-45d9-9077-7f6287a9d2a5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.755706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.756417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.756709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.756990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.769615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.771097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.781579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6736-360f-443e-a62f-d8222afc5c4e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:16.790280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:16.791221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:16.792157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:16.792157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:16.792157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:16.792157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:16.796067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 375/744] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:16.796937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.824976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:16.825901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:16.826228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:16.826462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:16.838028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:16.839526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:16.854867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:16.856249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:16.857247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:16.862736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 370/745] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:16.862736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:16.966175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:16.989585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a1e4-1c41-457a-8362-6ff555645cdf None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.032623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb97f13-10f8-430d-a62a-15c431c93f34 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.034112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.034904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.035185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.035462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.037003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:17.037318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.038703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.053694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 371/746] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:17.057631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.079786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.080398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.080680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.081803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.084076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.097600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.099949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:17.119459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:17.120534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 376/747] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:17.122061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.172968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e309ab-f4ea-431c-a897-5d588b29d727 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.220768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.222348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.222644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.222922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.223790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:17.224131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.226297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.254217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 372/748] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:17.254217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.254217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.255093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef458f57-0971-4d1b-9af6-2b0a34fdb319 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.312433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.313524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.313853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.314134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.324516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.326656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:17.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:17.343083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:17.343990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 377/749] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:17.344487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.378505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c0adf-b7f1-4095-9ff6-3b3ccf9bcb94 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.407605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.415713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.416418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.416701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.417020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.417881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:17.418061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.419418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.432669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 373/750] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:17.433685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.486579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b929-20b9-4395-99d3-a618b01aa202 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.501263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.531012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.531636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.531966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.532245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.549337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1c4c-1d87-4c3a-9f73-094b323c2192 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.595873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.596426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.597320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.597602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.598382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:17.598677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.600257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.618623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 374/751] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:17.619508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.622145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.627721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.690997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.751167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7505-2fa4-415e-b506-fcfff40c8fc1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.758295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 378/752] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:17.759358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.782940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.800212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.801109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.801536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.801956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.811049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.812704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:17.829468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:17.830594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:17.832148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:17.834760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 375/753] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:17.835887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.873259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698006ce-8142-4f3f-99e2-2b264e9f1a92 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:17.911948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:17.912775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:17.913182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:17.913464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:17.916541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=fe58a99358bc42aebeb126e5ef2cf21a, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:17.917970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:17.935914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 379/754] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:17 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:17.936916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:17.946371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:17.972759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85124a-4544-4081-97f9-0dfb4fcd5ed6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.012515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.013667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.014326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.014852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.027523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.029462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-04be05b2-797a-4f11-a980-c862e36f6bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:18.047152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:18.048413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:18.049314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:18.049314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:18.049314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:18.049314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.052243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 376/755] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:18.052665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c78af-bb14-49eb-ad37-a716567b4d89 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.054193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470f93-bf02-4cf2-aa3e-0e75648d1042 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.054787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470f93-bf02-4cf2-aa3e-0e75648d1042 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.055391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470f93-bf02-4cf2-aa3e-0e75648d1042 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.057608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 377/756] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:18.058719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5470f93-bf02-4cf2-aa3e-0e75648d1042 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.094877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.095521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.095806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.096116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.097118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:18.097433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.098993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7079a6-d914-427e-8385-2edc35005f30 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.120151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 380/757] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:18.121490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cba08-f316-4c1d-a132-497a833a164b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.121786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cba08-f316-4c1d-a132-497a833a164b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.122066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cba08-f316-4c1d-a132-497a833a164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.124474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.150263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-540cba08-f316-4c1d-a132-497a833a164b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:18.151790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-540cba08-f316-4c1d-a132-497a833a164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:18.199061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5470f93-bf02-4cf2-aa3e-0e75648d1042 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.232653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.242393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.243180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.243538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.243931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.244825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:18.245016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.245190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 381/758] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:18.246222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cba08-f316-4c1d-a132-497a833a164b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.254888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.276271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 378/759] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:18.280388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.373633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cba08-f316-4c1d-a132-497a833a164b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.412007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.412915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.413228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.413617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.414122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.422940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8a65-bb07-498e-9dfe-25aab53a77d1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.467026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.467803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.468125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.468398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] PATH_INFO: `/v3/services/network` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.469199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.471241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:18.483316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:18.484341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:18.485167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.485902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 379/760] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:18.485969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.486042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.498141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.499753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.602912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fc588-7261-466d-93de-da12c9c9a431 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.617835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 382/761] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:18.618798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.645054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.645698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.645975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.646246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.653491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.654952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:18.667097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:18.669137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:18.671237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.679901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 380/762] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:18.679901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.694589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.716463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.724570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606123d-e4d8-4dea-b5b3-11acdeee8209 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.761242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.761588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.761922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.762201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.762880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] RBAC: Adding query filter params (name=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:18.763184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.764586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.774763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 383/763] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:18.775708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.801844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962f2ba-91de-4637-9d32-a59e2ccbaee9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.837458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.838331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.838612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.838891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.850532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.852064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:18.867243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:18.868177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 381/764] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:18.873954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.881614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934758e0-7ea3-4733-9ff2-a7d83acafb48 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:18.915879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:18.916497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.916777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.917050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.917845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:18.918106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:18.919524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ffb8f-02ef-4208-a973-6393240ad767 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:18.931313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 384/765] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:18.932509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fb4a-f460-4a46-9382-1119b8dd1439 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:18.932793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fb4a-f460-4a46-9382-1119b8dd1439 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:18.933068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fb4a-f460-4a46-9382-1119b8dd1439 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:18.938081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 385/766] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:18.938081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71fb4a-f460-4a46-9382-1119b8dd1439 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:18.954680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.958861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:18.990192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c5d9d-ed0b-4b37-8f51-17da987d7f70 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.026654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.027504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.028006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.028550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.029419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] RBAC: Adding query filter params (type=network) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.029721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.031087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.037523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 382/767] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.038285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.055583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71fb4a-f460-4a46-9382-1119b8dd1439 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.092473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.093096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.093507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.093793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.094487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.094795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.096302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd83a1-32b4-432c-9b52-e97aa7546e25 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.114190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 386/768] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.114190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.114190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.114190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.130819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:19.132150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:19.150478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cc661-c52e-4a6c-adeb-bbb54c43bc56 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.185487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.186033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.186321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.186598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.187391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=6a40f282d2a94625ac21fb298a46b519) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.187703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.189171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.191523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.197136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 383/769] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/role_assignments?user.id=6a40f282d2a94625ac21fb298a46b519&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.198045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.214748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 387/770] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:19.216066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.229575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.348376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7ad10-b4ed-4f44-ad0b-99fb817eaf33 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.356441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef66-309f-4e5f-b313-b08b01d3c5e7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.386810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.387893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.388180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.388456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.395093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.395859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.396252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.396603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.397184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.397500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] RBAC: Adding query filter params (interface=public, service_id=987df0c94775476e83cdd2f742e51c5b, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.397879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.398907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.399482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.407440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 384/771] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/endpoints?service_id=987df0c94775476e83cdd2f742e51c5b&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.408310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:19.418052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:19.419030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:19.420165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:19.421698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 388/772] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:19.422953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.488191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.504384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.543225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a00b3-f9eb-4633-affd-66d6e84255f7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.544288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d179-8d7c-45fd-b2f0-c03e6c82abf8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.583854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.584626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.585002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.585374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.586238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.588012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.592610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.593426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.593857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.594306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.595149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.597263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.599588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 389/773] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.600672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:19.610334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:19.611545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:19.612662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:19.615319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 385/774] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:19.624121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.788342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca13f6b-a241-487b-bfbb-1ee0dca908bd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.813325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39be6-e6ae-45f8-b294-6c1df2a79bde None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:19.835385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.850215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.850946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.851232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.851510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.852235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.852631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.854140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:19.855390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.860913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:19.861534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:19.861820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:19.862122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:19.862337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 386/775] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:19.862876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:19.863183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:19.868682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:19.871038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:19.881993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 390/776] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:19.888618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.013528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300b354-c1bf-4c79-8a09-f8f931b6ad10 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.067201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ff0-84f8-4329-927e-24aaac11e1ab None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.084719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.085348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.085633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.085908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.086704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=eeec63e358424f57ae02d5693b359805) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:20.087025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.088501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b035747-3110-4135-ad68-975fb084d08b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.099552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 391/777] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/role_assignments?user.id=eeec63e358424f57ae02d5693b359805&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:20.101101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140bba-bf3f-4119-bfc3-b28949e5ef54 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.101530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140bba-bf3f-4119-bfc3-b28949e5ef54 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.101944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb140bba-bf3f-4119-bfc3-b28949e5ef54 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.110570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 392/778] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:20.121915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.122490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.122905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.127376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.128588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.128872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.129147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.129960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:20.130267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.131854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.148153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 387/779] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:20.149492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:20.149612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.150958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:20.217879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:20.246072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 393/780] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:20.257105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:20.294660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.324025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26736-7944-4a14-85e2-cc28664c7038 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.383363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.384046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.384397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.384669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.385535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] RBAC: Adding query filter params (interface=public, service_id=d8bf3c79dada427d815d469e52c7a7e7, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:20.385843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.387485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.401790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 388/781] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/endpoints?service_id=d8bf3c79dada427d815d469e52c7a7e7&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:20.412457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.438726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa8e51-e6fc-4514-b70c-ae3f4531b815 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.486690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.487924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.488215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.488493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.500644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:20.505930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:20.506854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:20.507692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:20.512075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 394/782] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:20.513320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.564687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:20.582784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dc3a2-8fc8-4915-a278-36dedb4673c2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.610548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:20.623544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2c52d-3f1f-4980-a98f-6169120e4193 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.644138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185523-569a-4606-87d2-55750d482645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.644763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185523-569a-4606-87d2-55750d482645 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.645041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185523-569a-4606-87d2-55750d482645 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.645314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4185523-569a-4606-87d2-55750d482645 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.645966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4185523-569a-4606-87d2-55750d482645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.647471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4185523-569a-4606-87d2-55750d482645 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.661385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.662331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.662597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 389/783] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:20.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.663681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.664708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:20.665223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.666839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.681499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 395/784] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:20.683228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185523-569a-4606-87d2-55750d482645 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:20.828902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4185523-569a-4606-87d2-55750d482645 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:20.884340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:20.885860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:20.886372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:20.886856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:20.904604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:20.906089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:20.922822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:20.923800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:20.925339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:20.925339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:20.925339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:20.925339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:20.928761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 390/785] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:20.932060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.049459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224a00-3d81-4455-a3a1-a85a58712b8d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:21.059401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:21.088844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:21.089478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.089764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.090038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.090737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:21.091045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:21.092452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:21.110067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 396/786] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:21.111442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.165903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:21.251426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b049ad-7470-4506-a177-ac89f868ca5d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:21.297263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:21.297904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.298185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.298458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.307520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:21.309026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b1d59ceb-0065-402a-9c4a-3d5cf31e82d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:21.325184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:21.326184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:21.327554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 391/787] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:21.334153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.461047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f84ee3-a58f-4d4d-aee0-9f215eacfbd1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:21.498680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaedca-8bb7-4c1e-8165-67d6909c0954 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.498680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaedca-8bb7-4c1e-8165-67d6909c0954 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.498680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaedca-8bb7-4c1e-8165-67d6909c0954 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.498680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 392/788] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:21.502920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:21.503436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.503717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.504020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.504442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.504923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.505420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.506432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:21.507022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:21.508686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:21.527372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 397/789] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:21.532555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.542946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:21.544760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:21.580197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:21.653381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 393/790] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:21.676450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.712002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:21.754647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b232-71cd-48ec-918d-dc93af4cf4f6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:21.798395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:21.799059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.799337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.799668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.806688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff531f3-7433-4c42-81d0-c1cf0b60e26b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:21.874251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:21.874973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:21.875251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:21.875521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:21.896677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:21.903473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:21.904417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:21.905262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:21.906682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 394/791] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:21.916250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:21.925244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:21.926685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bf1b5-9954-4c2e-918d-80a08b7c46ca None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:21.984186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.053813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.069790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de2026-293a-470f-843b-ea5c7693acb4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.081753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 398/792] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:22.083335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9afebe-7d00-43fa-8d90-13976e80e691 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.083860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9afebe-7d00-43fa-8d90-13976e80e691 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.084346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9afebe-7d00-43fa-8d90-13976e80e691 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.086069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 399/793] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:22.089614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9afebe-7d00-43fa-8d90-13976e80e691 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.120777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.122015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.122761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.123669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.124897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:22.125426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.127952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca830-45b0-47e3-b5f0-11c259570251 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.139188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 395/794] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:22.140743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.141237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.142302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.166982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:22.169420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:22.217116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9afebe-7d00-43fa-8d90-13976e80e691 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.251208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.263203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 396/795] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:22.264988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.273550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.274566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.275073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.275573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.292232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.294083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:22.320690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:22.321661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:22.322541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:22.322541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:22.322541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:22.322541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:22.325103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 400/796] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:22.326233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.366634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.391761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5d7f4-e2ee-47bb-9c37-65f2c3381cbb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.427990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.428632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.428911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.429183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.434086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1701-6903-4e0f-8b6c-ccc7cb9183f7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.439362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.440865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:22.457111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:22.458035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:22.458939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:22.458939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:22.458939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:22.458939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:22.458939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 397/797] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:22.459644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.473978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.474913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.475413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.475941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] PATH_INFO: `/v3/services/network` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.476766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.478406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:22.488199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:22.489197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 15:16:22.490030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:22.492750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 401/798] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:22.493730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.543280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.569955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f157-75e5-44fd-8132-d5b322ff5b5b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.597736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.605661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634444f6-f5ac-442e-a274-a17d53edff86 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.606236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.606956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.607259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.607535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.608246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:22.608561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.610882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.628277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 398/799] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:22.628277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.649654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.650282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.650568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.650845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.651538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:22.651883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.653240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.670325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 402/800] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:22.671178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.736907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba59bb-3657-40fc-8919-3f42ac24b37d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.772329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.772946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.773227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.773500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.774181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] RBAC: Adding query filter params (name=network) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:22.774484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.775865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9956e-92ad-45c0-b08a-f3c447a9be3b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.780221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4ef2-935c-47da-9ae7-5f26a9ce7984 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.780390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 399/801] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:22.781512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d7bb9-a215-42a2-9637-daf29d3d6810 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.781792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d7bb9-a215-42a2-9637-daf29d3d6810 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.782068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d7bb9-a215-42a2-9637-daf29d3d6810 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.783441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 400/802] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:22.784728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d7bb9-a215-42a2-9637-daf29d3d6810 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.785512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.824604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.825530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.826082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.827245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.835361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.837443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.844101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:22.854634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:22.855541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:22.856462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 403/803] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:22.856950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:22.898820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d7bb9-a215-42a2-9637-daf29d3d6810 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.932776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:22.933470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.933755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.934027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.941127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:22.942573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a7d1b906-3001-45e5-b089-572b412313b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:22.957478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:22.958628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:22.959763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:22.964922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 401/804] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:22.966328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:22.966611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:22.967267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:22.976644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1828-3f5c-4d8d-abf1-b96805d41f3b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:22.985578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:22.986769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:23.016619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.017160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.017438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.017711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.018394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] RBAC: Adding query filter params (type=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.018694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.020061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.029902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 404/805] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.030589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.060131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 402/806] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:23.061425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.072418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.115740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.154387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4808a98-f675-4f29-98bd-2fbcf0be2763 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.173305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f67e-1938-44b3-b775-0bc7eb9faed3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.189347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.189991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.190276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.190553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.191300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.191608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.193074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.205271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 405/807] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.206069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.208331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.208949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.209230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.209503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.210675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.210983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.212438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.229897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 403/808] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.230695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.290276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.305534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0c521-7aad-41d6-8212-7ab29f812c15 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.331354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.334218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7f4c-b78a-47b7-b7b0-0ba9566f7df5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.340298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.340917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.341196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.341468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.342173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.343530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c99b7-c2cc-468a-9902-af92079af9f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.366720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 406/809] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:23.368233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8f1f-4b06-42a8-9a16-f60cf2d60834 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.368526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8f1f-4b06-42a8-9a16-f60cf2d60834 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.368857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8f1f-4b06-42a8-9a16-f60cf2d60834 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.371886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 407/810] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:23.371886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d8f1f-4b06-42a8-9a16-f60cf2d60834 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.382661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.383336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.383618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.383923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.396745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:23.401951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:23.402985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:23.403814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 404/811] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:23.404648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.507572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d8f1f-4b06-42a8-9a16-f60cf2d60834 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.513610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080422f3-41c1-40f2-86e5-4dbe02266eb5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.547374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.548006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.548286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.548558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.549374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.549655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.551070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394d831-3876-4bb4-95df-36f6a76b4bd6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.555445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.556301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.556585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.556861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.558020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 408/812] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.559215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.559499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.559772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.568181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.569790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.574551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.583200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:23.585039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:23.588761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:23.589713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 405/813] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:23.591176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.606013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.675585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 409/814] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:23.675585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.727454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adeef0-d26f-4127-a0e3-2cf2359d574b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.771459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.772141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.772472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.773409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.773516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.773795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.775938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.784517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 406/815] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.785337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.799086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff96924-7e78-4719-a685-dd6cb295d74c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.834121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.834745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.835026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.835301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.836020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:23.836326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.837683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.840019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.853794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 410/816] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:23.853794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.864805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:23.899411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48578495-5658-4ea4-bd9f-f08d26f7665d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:23.936853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:23.937588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:23.937871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:23.938148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:23.938776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:23.940881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:23.950160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:23.951088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:23.951942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 407/817] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:23.952782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:23.967257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172ee17-8a48-4d4d-89b7-87e5d37a211b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.004550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.005187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.005466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.005737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.016894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.018875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:24.033801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:24.034719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:24.035605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:24.035605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:24.035605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:24.035605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:24.035605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 411/818] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:24.036501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.073539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd69dc-e0b5-41d5-ae79-6663a27e6383 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.088112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.110771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.111699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.112604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.112950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.114997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.120469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.122432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:24.134863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:24.135995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:24.137141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:24.138059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 408/819] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:24.138980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.151343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd8ce7-b129-473c-ba7a-ba2cbac52c87 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.187486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.188208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.188488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.188756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.189417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:24.189715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.191624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.196379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 412/820] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:24.197040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.263846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7fe16-07ac-41b1-b92f-919d179813c0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.296825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.297438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.297720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.297990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.298677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:24.298979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.300920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.311843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac396b9d-8da5-4c01-8f4b-aa3e45b1990c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.312458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 409/821] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:24.313266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.330921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.353554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.354394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.354897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.355323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.356267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:24.356824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.358363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.370310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.377402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 413/822] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:24.377402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.421666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d00c9c-c640-45a6-8993-c553ebf457d4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.458944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.459559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.459866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.460145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.460945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:24.461366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.462742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.472892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 410/823] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:24.472892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.481521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e219c-75af-4e39-af53-e7a6adfb2d1c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.515283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.515935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.516217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.516489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.523425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.524873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:24.537663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:24.538830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:24.539773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 414/824] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:24.540411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.584711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.584896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb92d-bf9d-46f1-92a0-9e8e3eced65e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.603097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.621420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.622079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.622357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.622632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/6a40f282d2a94625ac21fb298a46b519/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.636522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=6a40f282d2a94625ac21fb298a46b519, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.637965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.651523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 411/825] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:24 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/6a40f282d2a94625ac21fb298a46b519/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:24.652370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.671465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5fbb-ff0d-4819-aec1-c9f04144b418 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.726658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.727318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.727600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.727905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.728719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.730149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.762826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 415/826] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:24.788133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004186f8-8edf-478b-85d2-1ee09c3817aa None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:24.839368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:24.840328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:24.840808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:24.841260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:24.842279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:24.842775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:24.844456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:24.866270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 412/827] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:24.872046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:24.970610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:24.987124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061ddb-0383-4811-b27e-b4929a25cb3d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.022256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.022994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.023177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.023453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.035042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.036544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:25.051520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:25.052660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:25.053796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:25.053796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:25.053796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:25.053796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:25.055874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 416/828] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:25.061018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:25.087011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:25.169292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce537a11-48e1-42aa-a499-82b74958e4cb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.205243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.206735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.207166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.207587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.208528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:25.208978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.210514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.227004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 413/829] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:25.228622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:25.335231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cceb2-eb10-49e5-b61c-ad4ca99e0ae0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.371416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.372105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.372391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.372669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.379679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.381150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.383751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:25.393370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:25.394476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:25.395752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 417/830] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:25.402847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:25.490098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:25.511523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353546f-3e01-474b-b279-53934795bdc8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.546631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.547606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.548110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.548577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.550104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:25.550676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.552344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f079-c5f4-43b5-8a40-30ceecaf78fe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.565527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 414/831] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:25.575406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:25.649423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b58664-dc68-4d6a-a18b-2ef46fd333fa None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.649710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b58664-dc68-4d6a-a18b-2ef46fd333fa None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.649996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b58664-dc68-4d6a-a18b-2ef46fd333fa None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.651610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 415/832] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:25.656619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.656905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.657182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.676877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:25.678368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:25.686584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5ec4e-bf14-4df4-8cf0-704492daa4d7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.723212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.723917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.724206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.724490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/eeec63e358424f57ae02d5693b359805/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.726509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=eeec63e358424f57ae02d5693b359805, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.728100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.752009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 418/833] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:25 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/eeec63e358424f57ae02d5693b359805/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:25.764731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 416/834] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:25.789419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:25.860772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:25.915564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec63e1-6ad8-4606-abe1-d085fda65f1b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:25.951695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:25.952798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:25.953323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:25.953795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:25.954637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:25.956851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:25.966732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:25.967625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:25.968479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 419/835] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 15:16:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:25.973919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:26.050493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:26.083424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d110e-c02f-40d0-8698-803b53e96745 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:26.118833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:26.119458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.119742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.120046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.120729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:26.121036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:26.122424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeeedba-f80f-40a8-b307-57329258100b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:26.128019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 417/836] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:26.136923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:26.249105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c611d-2258-47bf-b2c9-be467f1ae623 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:26.277063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:26.288342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:26.288961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.289241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.289518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.290199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:26.290502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:26.291877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40eb31-9192-4aaf-94de-99d3292dd201 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:26.296961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 420/837] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:26.784598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4557-b11d-4202-b2c1-36c71d009887 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.784884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4557-b11d-4202-b2c1-36c71d009887 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.785164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4557-b11d-4202-b2c1-36c71d009887 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.786846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 418/838] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:26.792278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca345267-1974-4008-9a5a-261b69966e5b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.792573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca345267-1974-4008-9a5a-261b69966e5b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.792856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca345267-1974-4008-9a5a-261b69966e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.811688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca345267-1974-4008-9a5a-261b69966e5b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:26.812916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca345267-1974-4008-9a5a-261b69966e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:26.882096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 421/839] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:26.905021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe4557-b11d-4202-b2c1-36c71d009887 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:26.920347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfacaa-3cdc-4b1d-b8a9-520e600eb2e4 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.920347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfacaa-3cdc-4b1d-b8a9-520e600eb2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.920347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfacaa-3cdc-4b1d-b8a9-520e600eb2e4 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.921192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 422/840] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:26.926925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:26.927437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:26.927955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:26.947078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:26.951878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:27.023239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 423/841] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:27.034886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe4557-b11d-4202-b2c1-36c71d009887 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.048865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.078386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.079036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.079316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.079645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.083222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.092308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:27.098016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:27.098926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:27.099753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 419/842] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:27.105576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.112586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.164739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3729-b51d-4acb-a956-b2995fd62881 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.199860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.200581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.200871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.201150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.201780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.203354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.212863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7a85-cda0-4231-afcc-bc79fca4d6c9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:27.213485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:27.215446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:27.216502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 424/843] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:27.223880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.251004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.251624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.251936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.252238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.252957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:27.253262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.254641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.263368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 420/844] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:27.269335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.344900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.354092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d04f6-0908-4160-8858-cfb8bcb594d9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.372257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.395325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218782a-6273-4d65-ab4e-eec0ab1c0184 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.400391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.401025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.401312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.401595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.402285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:27.402597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.404009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.411808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 425/845] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:27.418042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.434803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.435720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.436221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.436671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.448865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.450655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:27.467615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:27.468610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:27.469488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:27.469488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:27.469488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:27.469488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:27.471552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 421/846] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:27.477470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.533701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98396-53ec-40db-894b-40c2dd1d5392 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.574658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.575285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.575663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.575989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.576773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:27.577082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.578546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.584492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 426/847] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:27.585988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352baebd-49f3-4a63-80b1-e64f9d05647a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.599145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.626571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.627780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.628105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.628382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.629101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:27.629407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.630892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.646043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 422/848] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:27.651787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.699672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:27.770816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79d315-d451-4413-b90f-4b11997bbfcc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.807604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.808277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.808554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.808826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.817592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.818986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3ef3d72d-4066-4974-bc14-a9b49ddb303f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:27.830890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:27.831907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:27.832863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 427/849] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:27.837987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:27.903476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c700b-b1df-42e4-9f08-ce0c80f48afd None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.903647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c700b-b1df-42e4-9f08-ce0c80f48afd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.903969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c700b-b1df-42e4-9f08-ce0c80f48afd None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.905546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 428/850] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:27.910609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.910889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.911164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.930106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:27.931307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:27.941698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca130e-23ff-48b1-8a70-2d2f090c7b72 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:27.976573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:27.977322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:27.977699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:27.978086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:27.978958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:27.979364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:27.980934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:27.999532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 423/851] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:28.005711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.023852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 429/852] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:28.033625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.040768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.088896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.115857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cd108-d77b-4677-b4ec-aebf756d42a6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.146500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.147118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.147399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.147668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.160796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76a860-e2b9-4dcd-b2f2-3bce9e293388 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.205269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.206072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.206359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.206932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.215649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.217147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.220577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:28.226539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:28.227572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:28.228457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 430/853] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:28.235355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.249838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.332103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 424/854] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:28.333140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.366430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.380665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7f9de-c924-4592-854b-be89aa51e5ea None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.423148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.423689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.424049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.424837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.425547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:28.425848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.427370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576c021-645e-4631-81ac-00cf52f6fca2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.436940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 431/855] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:28.436940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a02eb-d9d7-455c-a6d5-0391cb1ff24b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.436940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a02eb-d9d7-455c-a6d5-0391cb1ff24b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.436940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a02eb-d9d7-455c-a6d5-0391cb1ff24b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.436940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 432/856] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:28.442028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512830e4-b72f-454c-9ee9-be2a53f90e49 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.444817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a02eb-d9d7-455c-a6d5-0391cb1ff24b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.468690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.477802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.478438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.478721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.478994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.490100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.491560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2a2c5eff-5c8c-4c94-8496-ef8dc8a40b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:28.506074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:28.507124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:28.508216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:28.508216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:28.508216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:28.508216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:28.508216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 425/857] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:28.508672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4533-1290-459e-a851-875f61886657 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.508955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4533-1290-459e-a851-875f61886657 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.509232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4533-1290-459e-a851-875f61886657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.526555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-961f4533-1290-459e-a851-875f61886657 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:28.527730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-961f4533-1290-459e-a851-875f61886657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:28.560782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a02eb-d9d7-455c-a6d5-0391cb1ff24b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.594244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 426/858] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:28.595065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4533-1290-459e-a851-875f61886657 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.608126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.609263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.610177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.610638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.626023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.628002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:28.646797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:28.647701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:28.648634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:28.648634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:28.648634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:28.648634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:28.654004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 433/859] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:28.654004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.654211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.703716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4533-1290-459e-a851-875f61886657 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.715243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.741942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.742566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.742847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.743544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.744543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:28.744856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.746708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.755034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3095-89b0-4dd6-b086-6bedd398e15b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.761589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 427/860] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:28.761589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.796023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.796658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.796939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.797555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.797916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.799346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.820007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 434/861] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:28 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:28.821611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:28.880926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe24b-6c04-4413-b161-5eb6d6cc9b76 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.907658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:28.934798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:28.935720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:28.936169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:28.936689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:28.937737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:28.938265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:28.940623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:28.959757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 428/862] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:28.967329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85b25-5141-432f-8179-266461fbbe5a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:28.967462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.004125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.015382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.016607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.016607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.016607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.028688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.030190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:29.043066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:29.044006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:29.044936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:29.055970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 435/863] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:29.058291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.093346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e60c6d-60c3-4207-bf4f-a96c7f5b8978 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.137977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.139537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.139987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.140429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] PATH_INFO: `/v3/users/swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.150538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.152168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:29.164577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:29.165733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 15:16:29.166908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:29.168731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 429/864] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:29.174134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.185827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b69c9-bc0b-4cbb-b1b4-23bbf862ba19 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.228311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.234680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.235740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.236217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.237945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.239191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:29.239925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.242225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.259458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 436/865] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:29.265277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.267587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.286203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59623-9567-4685-bdaf-399eae54bcac None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.323436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.324535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.325024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.325502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.326506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:29.327023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.328750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.353262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 430/866] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:29.354890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.388893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51492ad-a9f8-46a7-a850-e79d82811845 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.434302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.434928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.435212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.435486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.436308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=6a40f282d2a94625ac21fb298a46b519) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:29.436619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.438042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf740b-f720-48e1-832c-cd8e7a7567b5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.451863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 437/867] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/role_assignments?user.id=6a40f282d2a94625ac21fb298a46b519&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:29.469522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4384970-cd71-4e5a-b792-a8491d94f8f0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.510658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.511209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.511491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.511767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.528955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.594344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.596580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.709318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21470-b239-49e9-9941-4e178ab3bd8a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.709597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21470-b239-49e9-9941-4e178ab3bd8a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.709876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21470-b239-49e9-9941-4e178ab3bd8a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.711224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 438/868] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:29.718397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.718749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.719099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.723860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 431/869] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 15:16:29.727519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.739394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:29.740731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:29.790098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.795430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:29.822318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 439/870] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:29 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:29.844863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.853130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dd283-083e-4d91-ac87-7907f1bf66a1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:29.896651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:29.897299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:29.897586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:29.898305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:29.911857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:29.913640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:29.932824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:29.933758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:29.935162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:29.935162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:29.935162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:29.935162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:29.935162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 432/871] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:29.941541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:29.959815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730f8f-52b9-4071-a15f-e123478b96c4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:30.011740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:30.012394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:30.012671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:30.012943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:30.013773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:30.014978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:30.042600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 440/872] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:29 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:30.072259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:30.080614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720af22b-9d94-4613-9cf9-53a8513f2f75 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:30.126230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:30.126854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:30.127809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:30.128113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:30.128869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:30.129177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:30.130571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:30.144323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 433/873] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:30.150588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:30.194687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:30.262290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f478e1-4812-4948-8b2d-f340435b87aa None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:30.297696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270d70a-8102-4960-804f-131f0615848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:30.298269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d70a-8102-4960-804f-131f0615848b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:30.298555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d70a-8102-4960-804f-131f0615848b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:30.299366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d70a-8102-4960-804f-131f0615848b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:30.306717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270d70a-8102-4960-804f-131f0615848b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:30.308214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270d70a-8102-4960-804f-131f0615848b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a270d70a-8102-4960-804f-131f0615848b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:30.320777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:30.321720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:30.322613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:30.326971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 441/874] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:30.328368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:30.437656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5e55-ed1f-487b-b339-016f18570c3b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:30.472903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:30.473524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:30.473804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:30.474074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:30.474824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:30.475126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:30.477002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76722a5c-1a98-4c77-9ba9-d9fd5777a012 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:30.490737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 434/875] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:30.501292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270d70a-8102-4960-804f-131f0615848b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:30.627729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:30.632755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270d70a-8102-4960-804f-131f0615848b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:30.671424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:30.672328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:30.672764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:30.673192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:30.674772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=eeec63e358424f57ae02d5693b359805) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:30.675237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:30.676831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3671ae-0528-4ba9-890f-8992b6bbedab None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:30.683694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 442/876] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:30 2022] GET /identity/v3/role_assignments?user.id=eeec63e358424f57ae02d5693b359805&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:31.005930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7d7e-d256-42c0-8e72-74dffe4be0fe None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.006220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7d7e-d256-42c0-8e72-74dffe4be0fe None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.006502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7d7e-d256-42c0-8e72-74dffe4be0fe None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.008150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 435/877] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:31.015153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.015453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.015709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.036085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:31.037603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:31.112930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 443/878] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:31.135051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f7d7e-d256-42c0-8e72-74dffe4be0fe None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.155460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:31.250069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f7d7e-d256-42c0-8e72-74dffe4be0fe None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:31.285015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:31.285655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.285934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.286209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.286835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:31.288236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:31.297320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:31.298263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:31.299727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 436/879] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:31.307322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.406144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:31.415036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83919f0d-a25a-4973-8bd4-d0a4ee663a4b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:31.451665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:31.452322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.452606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.452885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.453561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:31.453866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:31.455784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:31.461076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 444/880] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:31.466737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.545724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:31.571530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280af29-6f62-4a14-9a84-a851cdffc072 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:31.605171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:31.606111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.606588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.607057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.607973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:31.608473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:31.610069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872b35-770a-4ea5-a7a0-8a8348615b45 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:31.616008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 437/881] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:31.622032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.733845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ba8e-2123-40e6-8503-e035b32b54bf None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:31.739216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34b77-5b8a-4f8a-b2b2-df341b1aa19e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.739499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34b77-5b8a-4f8a-b2b2-df341b1aa19e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.739777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34b77-5b8a-4f8a-b2b2-df341b1aa19e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.741407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 438/882] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:31.746892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.747170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.747446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.766182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:31.767349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:31.776141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:31.777135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.777616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.778168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.779112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] RBAC: Adding query filter params (interface=public, service_id=23a9ce9d586f4af4990cb62e5674fba0, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:31.779654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:31.781415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:31.790204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 445/883] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/endpoints?service_id=23a9ce9d586f4af4990cb62e5674fba0&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:31.796007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.823904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:31.858241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 439/884] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:31.882584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:31.934805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44238c5-1eba-4596-bb83-400811904424 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:31.956689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:31.993626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:31.994247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:31.994527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:31.994801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:31.995487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:31.996834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.002463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f02fc-7b56-4f41-8045-b289c9219e1a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.012157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 446/885] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/services => generated 1268 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:16:32.043092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.044176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.044655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.045147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.058826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:32.064425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:32.065394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:32.066235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:32.069196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 440/886] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:32.075814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.175221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:32.193530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c5ed-8e4a-454f-b1ae-aa5e36be526c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.230008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.230629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.230908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.231182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.231918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:32.232229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.233577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8c341-d64a-44ae-a8ea-624358e87ea9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.243071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 447/887] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:32.248720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.275459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a783a1-7188-4fc2-ab82-c303186e1b27 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.275768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a783a1-7188-4fc2-ab82-c303186e1b27 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.276057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a783a1-7188-4fc2-ab82-c303186e1b27 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.277648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 448/888] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:32.282413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.282700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.282985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.302591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:32.303790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:32.362656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457025-9dba-4778-a629-67c37b50b0d9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.393614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:32.400795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.401737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.402288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.402787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.407904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 449/889] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:32.417501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.419201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.426554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:32.440873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:32.441794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:32.442673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:32.442673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:32.442673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:32.442673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:32.442673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 441/890] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:32.452321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.510243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:32.536582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5a127-1634-4581-b979-42ec23f7fc82 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.573319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.573982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.574260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.574530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.575145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.576527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:32.586169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:32.587199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:32.588437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 450/891] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:32.591575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51aa31-2ec4-40ec-90bd-b7a5bf50f625 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.593040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.628564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.629534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.630035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.630513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.631412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:32.631962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.633556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.647411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 442/892] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:32.655863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.707665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:32.711874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263998be-76e6-460c-8790-a55373ab7d41 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.749062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.749960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.750243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.750517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.751191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:32.751492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.753414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af78cd-13ea-4b3a-b956-fafe4ccf678e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.759869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 451/893] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:32.759869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d790-dffd-4d4c-9405-a5ff5492b658 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.759869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d790-dffd-4d4c-9405-a5ff5492b658 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.760158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d790-dffd-4d4c-9405-a5ff5492b658 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.761297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 452/894] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:32.765698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8d790-dffd-4d4c-9405-a5ff5492b658 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.770321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:32.777392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c4460-60e2-49b0-b148-39d08a271c0c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.815070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.815711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.816102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.816375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.824497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.825935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-655d8009-7ed0-4c94-a359-9608848c5248 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:32.838638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:32.839619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:32.840639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 443/895] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:32.841230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.841298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.841877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.859156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:32.860436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:32.883749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8d790-dffd-4d4c-9405-a5ff5492b658 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:32.921876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:32.922498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:32.922931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:32.923217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:32.924265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:32.924842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:32.926678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:32.936132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 444/896] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:32.936132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.936336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 453/897] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:32.937406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:32.976884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.020847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.050712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d11418-4b0b-4d13-bdb5-6ef190355f30 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.056385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee53954-bfb7-4cd2-adfc-fe90c2246ebc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.086599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.087221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.087505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.087783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.088611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:33.088922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.090355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.091233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.091870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.092152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.092426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.093619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] RBAC: Adding query filter params (interface=public, service_id=b796e016827f4d958c015ce46b817ba7, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:33.093619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.094801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.100733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 454/898] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/endpoints?service_id=b796e016827f4d958c015ce46b817ba7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:33.106081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.113587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 445/899] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:33.113587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.207906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.215745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922efc7-f31b-4b10-b0b7-856f2721e11b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.220264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91d29-dd33-43d2-a4da-46283232277f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.248792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.256282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.256925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.257210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.257489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.258117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.259477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.262097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.262715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.262999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.263277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.263967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.265350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:33.269164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:33.270132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:33.276527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 455/900] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:33.277344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 446/901] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:16:33.277418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.281231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.403667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98960e1-8e6d-4b7b-8c25-64208304f904 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.425018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abd6a2-48fa-4519-b418-4d5bbb7639dd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.445685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.446306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.446584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.446857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.447536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:33.448139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.449834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.455944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 447/902] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:33.462470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.469341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.470228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.471398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.471986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.485246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.544977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.556619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.558081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.586288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873968a1-cdbe-46ef-9d6a-e2ff3ba20296 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.629344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.630229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.630518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.630797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.631926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:33.632245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.633887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.640171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 448/903] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:33.691115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 456/904] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:33.692981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:33.800598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:33.812370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53fc53-6470-4ab5-9006-90630a051353 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:33.863139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:33.863776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:33.864083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:33.864522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:33.875072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:33.876553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:33.893989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:33.894932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:33.895852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:33.895852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:33.895852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:33.895852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:33.895852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 449/905] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:33.900986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.019939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.026417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7b83-8d4b-4bf3-9c88-759a44c047b1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.063210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.063812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.064124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.064427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.065117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:34.065422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.066780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12829c37-502c-40c5-ac11-43d7bd546e21 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.083968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 457/906] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:34.085484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.121368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb829e-2de7-4d82-a1d0-f836e718c81b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.121696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb829e-2de7-4d82-a1d0-f836e718c81b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.121928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb829e-2de7-4d82-a1d0-f836e718c81b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.123396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 458/907] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:34.129595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.129878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.130152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.149378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:34.150878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:34.216696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b75e9-03d7-48c9-8a3d-f737c6c29df7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.228137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 459/908] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:34.244942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.254415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.255060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.255343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.255618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.257849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.265204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.267073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:34.281576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:34.282445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:34.284290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:34.287891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 450/909] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:34.289732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.343336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.367472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b99d30-a773-41f2-8318-73065ac1073b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.415340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dce8ed-bae8-4793-b756-58bd87477e80 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.416424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.417583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.418950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.419414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.420362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.421915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:34.436079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:34.437462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 15:16:34.438291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:34.441667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 460/910] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:34.448712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.459215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.460361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.460660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.460937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.461696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:34.462004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.464002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.488862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 451/911] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:34.496854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.560009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.583581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabbd84-d878-40f5-ba4c-c9c4ffa19561 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.609520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009c342-2c8a-4d71-80ac-30b3563ab60d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.629483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.630113 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.630395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.630668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.631381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:34.632045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.634520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.646502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 461/912] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:34.648360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.657299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.658021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.658818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.659089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.659915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] RBAC: Adding query filter params (role.id=d4a05a85bf324fefae7801a6138c7d27, scope.project.id=3a947acc76624771ba3560291d433410, user.id=6a40f282d2a94625ac21fb298a46b519) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:34.660226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.661873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.667866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 452/913] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/role_assignments?user.id=6a40f282d2a94625ac21fb298a46b519&scope.project.id=3a947acc76624771ba3560291d433410&role.id=d4a05a85bf324fefae7801a6138c7d27 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:34.671250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.774132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9265d-c1f4-41d8-958c-2ed47d95f140 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.812152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.813200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.813746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.814267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.815196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:34.815712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.817546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973cce-f6a0-434d-9e59-b3150a162e4a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:34.823979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 462/914] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:34.832980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:34.937536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11580-7b38-444b-ad52-ccf92cb45ed9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:34.978586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:34.987392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:34.988049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:34.988758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:34.989385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:34.990543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:34.991949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf527e-404c-4d6f-ba3a-005050392fd8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:35.023976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 453/915] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:35.438351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9659-2d88-4992-8bbc-753ccba024c9 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.438902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9659-2d88-4992-8bbc-753ccba024c9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.438902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9659-2d88-4992-8bbc-753ccba024c9 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.441086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 463/916] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:35.446295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.446711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.446883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.465383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:35.466572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:35.541009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 454/917] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:35.563285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9659-2d88-4992-8bbc-753ccba024c9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:35.644560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:35.667877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9659-2d88-4992-8bbc-753ccba024c9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:35.704485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:35.705089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.705373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.705641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.706271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:35.707646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-307162e7-972b-468a-9eb3-719809bcc425 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:35.717260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:35.718246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:35.719082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 464/918] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:35.727969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:35.740304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d8-e1ad-4e1f-9034-dba0b8c2421e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.740605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d8-e1ad-4e1f-9034-dba0b8c2421e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.740865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d8-e1ad-4e1f-9034-dba0b8c2421e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.742437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 465/919] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:35.747322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.747611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.747912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.770763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:35.772046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:35.843594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 466/920] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:35.855617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ae743-f023-4e4d-98b5-40ed9e379a36 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:35.867145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:35.868089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:35.898306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:35.899350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:35.899634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:35.900197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:35.900889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:35.901448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:35.903540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:35.911338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 455/921] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:35.915884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:35.949775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:35.984626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943efa8-2011-47a8-bf18-525f59bc362b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:36.017961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfce9cc-adad-428e-92d4-a46cd3624ee6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:36.022156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:36.022777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:36.023060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:36.023332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:36.024048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:36.025491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:36.047978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 467/922] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:35 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:36.068242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:36.069182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:36.069685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:36.070193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:36.071123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:36.071654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:36.073629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de202587-3df1-40d0-bf82-efba9691b8e9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:36.081144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 456/923] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 15:16:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:36.087556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:36.200673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c153-4a48-430e-b07c-e95feb485fbb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:36.238047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:36.243695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:36.244348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:36.244676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:36.244952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:36.245718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:36.247458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0775b9-9d14-4177-b0ec-51dfac923ec2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:36.270424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 468/924] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:36.802970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7cdd-f3de-44a9-9175-f136971b47ab None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:36.803517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7cdd-f3de-44a9-9175-f136971b47ab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:36.803517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7cdd-f3de-44a9-9175-f136971b47ab None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:36.805292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 457/925] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:36.810523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f5ca-529b-4838-b593-ec850ab7ddf8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:36.810916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f5ca-529b-4838-b593-ec850ab7ddf8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:36.811155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f5ca-529b-4838-b593-ec850ab7ddf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:36.830440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5753f5ca-529b-4838-b593-ec850ab7ddf8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:36.831635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5753f5ca-529b-4838-b593-ec850ab7ddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:36.911939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 469/926] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:36.928813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff7cdd-f3de-44a9-9175-f136971b47ab None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:36.965846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.053207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff7cdd-f3de-44a9-9175-f136971b47ab None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.083373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d2966-337a-4335-b490-df00ac328483 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.083693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d2966-337a-4335-b490-df00ac328483 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.083970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d2966-337a-4335-b490-df00ac328483 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.085445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 470/927] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:37.090780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.091067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.091353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.092579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.093452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.093924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.094487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.105810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:37.109709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:37.111421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:37.112129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:37.113020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:37.113843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:37.116523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 458/928] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:37.128466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.186729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 471/929] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:37.208585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.224192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.248493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2dc4-a874-444c-88dc-659b99056099 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.262128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.296870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.297521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.297807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.299174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.300017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:37.300335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.301785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.313453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 459/930] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:37.318949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.343103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57b74-27ea-404a-99cf-376790e08fe2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.387241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.388455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.388951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.389872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.390968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.393304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:37.404829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:37.405868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:37.406734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:37.414301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 472/931] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:37.415363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.426688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a84a1a-6b5c-4c64-b157-3e24f11046e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.468926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.469702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.470096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.470500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.483337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.484825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.491778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:37.502459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:37.503364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:37.504840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:37.504840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:37.504840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:37.504840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:37.508425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 460/932] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:37.509672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.553778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2997f54-1eb7-4ba7-92a7-6b4cf65f9082 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.566999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.602342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.603194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.603651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.604110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.605426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:37.606155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.607980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.616212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 473/933] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:37.622109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638e0b-4946-4884-b169-59009c7de86b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.624653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.658671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.659698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.660238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.660762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.661652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:37.662227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.663945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.681928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 461/934] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:37.683063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.746978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:37.749681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d99b8-d387-4a73-ac68-e52b1ddd7d60 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.801378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.802570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.803049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.803532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.804532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] RBAC: Adding query filter params (type=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:37.805033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.806650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.816836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda383c6-ecb0-4822-82b2-1436792394d5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:37.817902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 474/935] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:37.851189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:37.851899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:37.852188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:37.852464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:37.860739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:37.862781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:37.876112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:37.877691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:37.878619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 462/936] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:37.885521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:37.933634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.020932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6c43-6268-4231-86b0-3d8453faf6da None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.058285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.058901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.059186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.059460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.060298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:38.060619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.062021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4ff1d-996e-4c98-9ff9-6179f91550e7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.084095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 475/937] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:38.092245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.198706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775fef-2207-4422-9611-4973c4d4dac8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.216420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.225852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c917-571f-4a07-a125-9cd5030b3ec4 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.226242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c917-571f-4a07-a125-9cd5030b3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.226408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c917-571f-4a07-a125-9cd5030b3ec4 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.228035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 476/938] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:38.233156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b1c5-72be-4684-a105-d65d544d1d8c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.233446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b1c5-72be-4684-a105-d65d544d1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.233728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b1c5-72be-4684-a105-d65d544d1d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.235072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.235767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.236639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.237001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.248368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.249757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.252529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85a1b1c5-72be-4684-a105-d65d544d1d8c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:38.253798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85a1b1c5-72be-4684-a105-d65d544d1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:38.264834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:38.265905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:38.266919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:38.266919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:38.266919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:38.266919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:38.266919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 463/939] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:38.271975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.348292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 477/940] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:38.363140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d15b1-1b8f-4a47-a960-bfd635fcb80f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.363429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d15b1-1b8f-4a47-a960-bfd635fcb80f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.363722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d15b1-1b8f-4a47-a960-bfd635fcb80f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.365087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 478/941] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:38.365785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d15b1-1b8f-4a47-a960-bfd635fcb80f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.396407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51f1-4f8c-4959-b38f-65dfb33297ae None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.397263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.435480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.436519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.437011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.437466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.438332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:38.438817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.444212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20390dde-e1dc-4937-87f6-2a99409b2be5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.458636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 464/942] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:38.460153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.460440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.460716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.478459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:38.479884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:38.480616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d15b1-1b8f-4a47-a960-bfd635fcb80f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.503896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.516478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.517689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.517976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.518256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.518883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.520359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:38.529774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:38.530786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.532309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 479/943] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:38.533751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.554189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 465/944] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:38.562042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.634824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.648295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd014c8c-4e6b-498a-beaa-80cdc9c972dc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.672803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e82f2-6b1a-43d8-ab6e-c4915261c179 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.683547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.684221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.684503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.684781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.692365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.693845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:38.706765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:38.707639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.708637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 480/945] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:38.709149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.712621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.713255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.713537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.713820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.715224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:38.715224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.716197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.721049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 466/946] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:38.721829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.753401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.824883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fdcd4-033f-4a7f-ace2-570c57fe00f6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.831695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0785-0c9b-42cd-afaf-485a24ced413 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:38.861632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.862256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.862600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.862840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.863486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.864936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.867535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING keystone.server.flask.common [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 15:16:38.869426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:38.870048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:38.870333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:38.870613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:38.871362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:38.871681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:38.873133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:38.890304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 467/947] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:38.890304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.893751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 15:16:38.893751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 15:16:38.893751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [parameters: {'id': '267a296607dd40d2a4eb01ff435ed7c4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 15:16:38.893751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 267a296607dd40d2a4eb01ff435ed7c4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 267a296607dd40d2a4eb01ff435ed7c4. Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:16:38.901607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 15:16:38.902568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 27 15:16:38.903734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 15:16:38.904970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 15:16:38.906070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:16:38.907029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'id': '267a296607dd40d2a4eb01ff435ed7c4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.907954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:38.908751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 267a296607dd40d2a4eb01ff435ed7c4. Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 481/948] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 15:16:38 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:38.909959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:38.912028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:38.996772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:39.016466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9969d52-d009-4e75-9e64-c286a88c6d2a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.027136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038d408-5752-4163-a797-5eb0294bb49f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.062902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.063522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.063802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.064765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.065442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:39.065744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.066146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.066842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.067114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.067206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.067388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/6a40f282d2a94625ac21fb298a46b519/roles/d4a05a85bf324fefae7801a6138c7d27` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.069691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=6a40f282d2a94625ac21fb298a46b519, role_id=d4a05a85bf324fefae7801a6138c7d27)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.071275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.077187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 468/949] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:39.077187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:39.095893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 482/950] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:38 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/6a40f282d2a94625ac21fb298a46b519/roles/d4a05a85bf324fefae7801a6138c7d27 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:16:39.095893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:39.185277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:39.213340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680b73-efff-46b9-b708-a359c0549492 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.241737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1718-50db-4d0f-99a8-62e4fd42cf0d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.256793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.257610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.257989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.258410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.267145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.268958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:39.281897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:39.283001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:39.284146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:39.284682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.284682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.284682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.284682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.284991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] RBAC: Adding query filter params (interface=public, service_id=64c8118e4ec64a6b824b96ba222a07cc, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:39.284991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.287269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.287391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 469/951] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:39.292084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:39.295939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 483/952] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:39 2022] GET /identity/v3/endpoints?service_id=64c8118e4ec64a6b824b96ba222a07cc&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:39.300791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:39.306340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:39.403003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42890a-c42e-46a3-abb5-58a0775e511c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.411862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada0c5-3cfe-4fbe-ae87-9fc0574e2221 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:39.439736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.440308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.440594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.440871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.441110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:39.441854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:39.441975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.443330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208113a3-16db-4ffa-b4d6-9a8eadaab44f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.453061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:39.454033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.454532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.455014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.455883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:39.457431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd403d-3b28-4719-9e20-290c7ab76f6a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:39.472169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 470/954] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:39.472381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 484/954] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:39 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:16:39.976294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc75c7-6270-40b5-a78f-0195080fc78e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.976832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc75c7-6270-40b5-a78f-0195080fc78e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.976832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc75c7-6270-40b5-a78f-0195080fc78e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:39.978491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 471/955] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:39.983894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52866f0b-8afb-437b-b8b9-5077db8f1b24 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:39.984335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52866f0b-8afb-437b-b8b9-5077db8f1b24 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:39.984510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52866f0b-8afb-437b-b8b9-5077db8f1b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:40.003450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-52866f0b-8afb-437b-b8b9-5077db8f1b24 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:40.005793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-52866f0b-8afb-437b-b8b9-5077db8f1b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:40.079090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 485/956] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:40.101404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc75c7-6270-40b5-a78f-0195080fc78e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:40.175453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:40.212547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc75c7-6270-40b5-a78f-0195080fc78e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:40.248905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:40.249462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:40.249749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:40.250030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:40.250737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:40.252232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffc9eb-bb08-472c-91bb-79afa86753cb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:40.273298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 472/957] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:40 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:41.180726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7dc3-3ceb-4922-81ef-40e663d3768f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.181265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7dc3-3ceb-4922-81ef-40e663d3768f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.181265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7dc3-3ceb-4922-81ef-40e663d3768f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.182893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 486/958] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:41.188445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.189407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.189588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.208520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:41.209720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:41.285881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 473/959] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:41.308948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add7dc3-3ceb-4922-81ef-40e663d3768f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.417242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add7dc3-3ceb-4922-81ef-40e663d3768f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:41.445602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:41.459406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:41.460375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.460888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.461364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.472723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5525750d-b079-4926-927b-5b3fb78da526 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:41.478020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:41.479007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:41.479928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.482773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 487/960] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:41.489031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.497382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edefc-7e1f-4938-8210-b0ea55e5e49d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.497676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edefc-7e1f-4938-8210-b0ea55e5e49d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.497965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edefc-7e1f-4938-8210-b0ea55e5e49d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.499540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 488/961] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:41.504899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328174a8-05d4-4740-bac8-665d1b910b25 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.505187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328174a8-05d4-4740-bac8-665d1b910b25 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.505470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328174a8-05d4-4740-bac8-665d1b910b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.523688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-328174a8-05d4-4740-bac8-665d1b910b25 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:41.524930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-328174a8-05d4-4740-bac8-665d1b910b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:41.550036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:41.603932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 489/962] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:41.604149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36933de5-d973-4455-a5d4-d8af421608d7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:41.619078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745a042-99c1-427c-8f52-64d667dfa322 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.619366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745a042-99c1-427c-8f52-64d667dfa322 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.619642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745a042-99c1-427c-8f52-64d667dfa322 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.621239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 490/963] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:41.624791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745a042-99c1-427c-8f52-64d667dfa322 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.646184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:41.646801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.647083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.647359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.648284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:41.648596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:41.649971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b137-e2fc-4457-b431-4567f6241bed None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:41.658946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 474/964] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:41.660246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.660566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.660804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.680459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:41.681751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:41.738599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745a042-99c1-427c-8f52-64d667dfa322 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:41.751847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:41.760301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 475/965] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:41.760301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.775162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:41.775992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.776315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.776601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.777246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:41.778751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:41.789323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:41.791298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:41.792926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.796104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 491/966] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:41.796841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.827614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:41.884885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bb916-c9ac-4ba0-84f1-c8e28fb31565 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:41.898755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaee59-10ab-455f-b865-7783627fe330 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:41.922667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:41.924245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.924686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.925097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.934095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:41.934716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:41.934999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:41.935273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:41.935971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:41.936591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:41.937390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:41.938320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:41.955216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:41.956290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:41.957186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:41.957186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:41.957186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:41.957186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:41.957435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 15:16:41.957435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 15:16:41.957435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [parameters: {'user_id': '35d6b4427e664a5f93a3c7fe92b06868', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 15:16:41.957435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 15:16:41.963209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 476/967] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:41.963209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:16:41.963532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:16:41.964455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 15:16:41.965478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 15:16:41.966406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 15:16:41.967323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 15:16:41.969632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'user_id': '35d6b4427e664a5f93a3c7fe92b06868', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:41.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 15:16:41.975220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 492/968] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 15:16:41 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:41.979696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.017248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.063876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.068380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb89-a9e9-4742-9c9f-d1b3e8b05ee5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.085596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badc547-8647-4aa5-a885-8378a7028205 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.106083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.106796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.107154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.107501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.108291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:42.108718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.110208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.115244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 477/969] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:42.115244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.127845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.128796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.129103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.129520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.130213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:42.130519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.132106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.147517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 493/970] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:42.148357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.228248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2175-a365-43ee-bff9-2b549563ac6f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.251324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.256781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc345-e2c1-446d-ad4b-0d7b1aa40ad3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.263677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.264349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.264629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.264902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.273497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.274837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:42.286702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:42.287701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 478/971] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:42.288714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.300542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.301172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.301456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.301736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.302430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:42.302738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.304111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.309598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 494/972] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:42.310386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.317762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.396544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50342c3f-69d2-418f-8084-568d7b8dad4b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.420185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026fc22-62b5-4ce7-a625-903e98a3670d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.429255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.429888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.430178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.430450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.437738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.439739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:42.451055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:42.452197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 479/973] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:42.453317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.457153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.457971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.458255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.458526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.459281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:42.459581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.461011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b291-1355-4240-b89b-0d75c71076e7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.494840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 495/974] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:42.496266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdac34-78f4-4ecf-a32f-6b3f035fb9d7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.496550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdac34-78f4-4ecf-a32f-6b3f035fb9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.496825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdac34-78f4-4ecf-a32f-6b3f035fb9d7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.498202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 496/975] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:42.498844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdac34-78f4-4ecf-a32f-6b3f035fb9d7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.532203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.544840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.567699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fc858-08ab-4e95-8d2a-54e7f3d89ce4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.605369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.606004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.606286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.606560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.607277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.608701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329110e-3aeb-4048-a937-bfa138d16fc2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.627268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdac34-78f4-4ecf-a32f-6b3f035fb9d7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.631902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 480/976] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:42.633028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.633315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.633611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.671665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:42.674103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:42.694775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.695409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.696053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.696053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.696848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:42.697164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.700716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.721528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 497/977] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:42.728214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.777120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 481/978] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:42.801870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.819320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.848752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd73-2cbb-4f8f-8dc6-68ee27b68af8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.873746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:42.888920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.889550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.889836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.890111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.911429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ade59-ce96-4334-ad8a-0634050f55cf None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:42.945855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:42.947885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:42.947885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:42.947885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:42.947885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.949269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:42.960444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:42.961669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:42.962718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:42.965368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 482/979] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:42.971137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:42.977673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:42.979184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.058100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.081563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117e8b-012b-498d-841d-a5bedd709849 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.087388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 498/980] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:43.092708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.120906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.128644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.129342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.129688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.130017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.130759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.131129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.132621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.142835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 483/981] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:43.143579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.203176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3049-d836-470d-86fe-de9e52536f01 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.242201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.242947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.243226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.243505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.248526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba21f-85d9-4236-b142-1ba90b4ad205 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.255030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.256611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:43.272682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:43.273784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:43.274830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:43.274830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:43.274830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:43.274830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:43.274830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 499/982] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:43.279516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.284422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.285104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.285447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.285813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.286556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.286926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.288400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.296181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 484/983] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:43.302498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.309192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.366342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.387902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d780d5-e514-4e69-bf04-43334285cace None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.410939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c25d7a-805a-4f00-8126-ea5506846385 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.426393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.427089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.427443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.427790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.428583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.428963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.430398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.442795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 500/984] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:43.448444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.449821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.450596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.450982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.451356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.452247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] RBAC: Adding query filter params (interface=public, service_id=e7ad9bfaa5cb4858af7c756ff7860710, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.452659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.454169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.463858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 485/985] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/endpoints?service_id=e7ad9bfaa5cb4858af7c756ff7860710&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:43.464890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.541151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.558157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea703-5d9f-4175-9696-d38c84b67f13 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.576146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a39-2c87-4315-a37a-e8a535c8bd94 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.594097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.594736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.595593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.595895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.602736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.603663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:43.604197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:43.615699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:43.616711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:43.617634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:43.618087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.618087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.618087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.618087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.618362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.618918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.620119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 501/986] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:43.622880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.636109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 486/987] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:16:43.763915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b853-2306-4f3e-b715-30cf76413224 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.803030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.803762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.804079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.804356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.805121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.805433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.806869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a42e3-1ca1-4373-ba80-4fb4a8c7d2b1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.826632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 502/988] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:43.833469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:43.940134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d3fa-528b-4aa0-888b-0c6c82e9bcc4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:43.975530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:43.976176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:43.976461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:43.976737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:43.977526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] RBAC: Adding query filter params (role.id=d4a05a85bf324fefae7801a6138c7d27, scope.project.id=3a947acc76624771ba3560291d433410, user.id=6a40f282d2a94625ac21fb298a46b519) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:43.977837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:43.979264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5eee6-f8fe-4879-a536-99564cb1edd8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:43.992010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 487/989] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:43 2022] GET /identity/v3/role_assignments?user.id=6a40f282d2a94625ac21fb298a46b519&scope.project.id=3a947acc76624771ba3560291d433410&role.id=d4a05a85bf324fefae7801a6138c7d27 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:44.671199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698afcb-d0a9-40b0-be2b-1034055573e0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:44.671726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698afcb-d0a9-40b0-be2b-1034055573e0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:44.671726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698afcb-d0a9-40b0-be2b-1034055573e0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:44.673285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 503/990] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:44.678267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:44.678645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:44.678819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:44.697540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:44.698738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:44.748885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:44.771292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 488/991] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:44.792987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698afcb-d0a9-40b0-be2b-1034055573e0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:44.802565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:44.898518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698afcb-d0a9-40b0-be2b-1034055573e0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:44.931314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:44.931911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:44.932192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:44.932469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:44.933089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:44.934450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:44.943935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:44.944919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:44.945760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:44.947920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 504/992] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:44.955879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:45.054283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca7222-d526-45fb-8b6b-107e79ff828b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:45.088806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:45.089425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.089707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.089982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.090657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:45.090962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:45.092366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:45.097321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 489/993] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:45.103145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:45.175306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:45.207922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d9b9-6885-41e5-bd32-8e79ffe89542 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:45.240951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:45.241564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.241841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.242113 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.242775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:45.243076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:45.244561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:45.249500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 505/994] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:45.256004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:45.304904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:45.358196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452381c-2e6d-4c74-a7f4-01879c2df6a3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:45.392213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:45.392828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.393111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.393387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.394152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] RBAC: Adding query filter params (interface=admin, service_id=64c8118e4ec64a6b824b96ba222a07cc, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:45.394461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:45.395843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4ca1f-786d-4c91-a0f7-70d55c2f85cc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:45.405247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 490/995] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Jul 27 15:16:45 2022] GET /identity/v3/endpoints?service_id=64c8118e4ec64a6b824b96ba222a07cc&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:45.406796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:45.509896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a116-10d3-4060-aa8d-f492ec32aa62 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:45.542819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:45.543429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.543711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.544012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.544650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:45.546003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9d062-0201-4df6-926d-a8d19a5c4ba0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:45.547056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:45.555998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 506/996] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:45 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:16:45.731349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d01d6-e723-4e7c-9cc2-0779d1833c6f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.731899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d01d6-e723-4e7c-9cc2-0779d1833c6f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.731899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d01d6-e723-4e7c-9cc2-0779d1833c6f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.733398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 491/997] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:45.739067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.739475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.739658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.758105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:45.759614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:45.827788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 507/998] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:45.850918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d01d6-e723-4e7c-9cc2-0779d1833c6f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:45.955443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d01d6-e723-4e7c-9cc2-0779d1833c6f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:45.967809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:45.993200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:45.993844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:45.994122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:45.994395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:45.995018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:45.996418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-41dc85fe-8a4f-4cc9-895e-10fb8b4876f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:46.005570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:46.006558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 15:16:46.007487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.011916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 492/999] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:46.013131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.041442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a71c5-7240-4762-b9d4-d490f4ed6d28 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.041719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a71c5-7240-4762-b9d4-d490f4ed6d28 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.041997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a71c5-7240-4762-b9d4-d490f4ed6d28 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.043461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 493/1000] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:46.049083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.052914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.053219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.076125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:46.077332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:46.124047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d8fff-e2ae-425a-8eb4-f1b946f7ba27 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.148411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 494/1001] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:46.154418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:46.163595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.164263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.164543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.164817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.165493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:46.165795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.167149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.171230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.176909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 508/1002] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:46.177723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.276779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a725d9a-a4c4-4120-b343-bf7d1a77ffa1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.281206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:46.286991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620da9-3973-4a5c-bf17-3019f1152790 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.316684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.317229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.317510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.317946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] PATH_INFO: `/v3/services/image` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.318410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.319920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.323055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.323486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.323763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.324154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.324720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:46.325019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.326341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:46.330234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:46.331226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:46.332080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 495/1003] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 15:16:46 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:46.341844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 509/1004] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 15:16:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:46.341984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.342453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.401083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:46.445669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb84ac-cab5-40f0-a305-0743d35ee460 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.448835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8248f4-325b-4251-89fa-861c54fde066 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.480343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.481064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.481414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.481757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.482503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] RBAC: Adding query filter params (name=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:46.482878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.484211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.484426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.484964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.485334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.485684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.486381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.487724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fdec5-7e01-48e5-a721-e78142f3e917 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.494970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 496/1005] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:46.495796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:46.514238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 510/1006] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:46.538760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:46.619537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5ac83-f313-4140-b1fb-24b09adfad4a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:46.663349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:46.664361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:46.664361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:46.664538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:46.665778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:46.665778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:46.666951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d372e-9b56-4a68-bd6c-a6235d7651d0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:46.674584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 497/1007] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:47.555692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7bdd-70eb-4689-aa4b-07c8c71f8b15 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:47.556276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7bdd-70eb-4689-aa4b-07c8c71f8b15 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:47.556276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d7bdd-70eb-4689-aa4b-07c8c71f8b15 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:47.557740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 511/1008] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:47.562740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:47.563131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:47.563290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:47.581848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:47.583018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:47.649597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 498/1009] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:47 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:47.672051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d7bdd-70eb-4689-aa4b-07c8c71f8b15 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:47.783969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d7bdd-70eb-4689-aa4b-07c8c71f8b15 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:47.796960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:47.824239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:47.824802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:47.825082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:47.825347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:47.825973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:47.827338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:47.839824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:47.840890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:47.841800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 512/1010] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 15:16:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:47.847275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:47.873027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:47.950676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c458e-fbc8-4f9c-9723-d64edd8ec80f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:47.987444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:47.988087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:47.988369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:47.988640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:47.989311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:47.989609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:47.990967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:47.995962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 499/1011] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:48.002091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:48.100974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7eb2ad-7500-43c3-9a9f-f1484a4194b3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:48.134949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:48.135562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.135864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.136136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.136800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:48.137099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:48.138442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a7b7-7fb9-423c-8d22-df9c908edf00 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:48.143404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 513/1012] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 15:16:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:48.149099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:48.234817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:48.251821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924fb-ed15-45d7-b523-9d9f4ad42939 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:48.285212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:48.285831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.286111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.286386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.287087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:48.288483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7a8f-2de4-4bba-9556-d594566f1a17 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:48.311868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 500/1013] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:16:48.652847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c9fdb-cc50-44fe-a672-4cc1c963dc3f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.653402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c377-21a9-41cd-9c0a-82fc86ee2765 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.653756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c9fdb-cc50-44fe-a672-4cc1c963dc3f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.653836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c377-21a9-41cd-9c0a-82fc86ee2765 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.653907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c9fdb-cc50-44fe-a672-4cc1c963dc3f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.653977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c377-21a9-41cd-9c0a-82fc86ee2765 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.654900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 514/1014] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:48.655053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 501/1015] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:48.660324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.660462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.660642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.660824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.660942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.661099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.679426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:48.679608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:48.680645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:48.680775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:48.719332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:48.757605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 502/1016] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:48.767867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 515/1017] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:48 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:48.780745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:48.788641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:48.856577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:48.887905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb223-2ce8-48e0-9712-7bc65eb24069 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:48.890271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d712bcd-c418-4234-a862-6e2f635b48ac None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:48.923684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:48.924260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.924545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.924818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.925512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:48.926868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:48.927460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:48.928179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:48.928469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:48.928746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:48.941498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:48.945461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 516/1018] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:48 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:48.949140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:48.950091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:48.950979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:48.955392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 503/1019] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:48.955539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:48.992221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:49.082637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf8201-2f49-4cf3-9882-22cdbf1139aa None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.121566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.122193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.122478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.122755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:49.123477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:49.123786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:49.125191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:49.134320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 517/1020] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:49.139724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:49.242399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2996-7863-4a0d-8d39-f4bab0762fa3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.272571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:49.279009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.279651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.279971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.280250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:49.290616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:49.292103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:49.306789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:49.307712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:49.308911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:49.308911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:49.308911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:49.308911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:49.309495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 504/1021] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:49.314329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:49.400585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:49.418151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cf7dd-6163-46e7-9a5c-f285bc22e548 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.456696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.457312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.457591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.457862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:49.458541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:49.458844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:49.460218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:49.476090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 518/1022] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:49.477351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:49.581604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621769-95a8-41fb-af8a-37301c9d228b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.614705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.615344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.615625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.615929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:49.623301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:49.624804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:49.636434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:49.637350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:49.638299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:49.639422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 505/1023] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:49.643427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:49.744976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f44e-71a7-4907-b340-ce5e67691e7d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.778967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.779590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.779899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.780182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:49.780973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:49.781284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:49.782728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:49.806116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 519/1024] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:49.807361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:49.861238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:49.918090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba992e-db71-45e3-8973-96072bc63642 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:49.955104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:49.956002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:49.956425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:49.956831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.028333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.029786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.074519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.139142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 506/1025] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:50.140288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.143162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.244736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dba74d-415d-4e42-8891-44410f9c8e4d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.278510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.279148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.279432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.279707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.290463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.291945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e9255c73-b459-4eb1-9b61-157b89026b00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:50.307957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:50.308851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:50.309735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:50.309735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:50.309735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:50.309735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:50.310510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 520/1026] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:50.314982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.323943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799e4d-06ec-4468-acb1-7d331496ade0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.324230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799e4d-06ec-4468-acb1-7d331496ade0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.324503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799e4d-06ec-4468-acb1-7d331496ade0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.325833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 521/1027] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:50.331140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.331422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.331693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.349916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:50.351090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:50.401339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.419137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5e4b-d00f-4389-b854-5f3416eb1244 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.421517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 522/1028] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:50.443908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.454605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.455220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.455497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.455769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.456483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:50.456783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.458149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.473686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 507/1029] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:50.478830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.482338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.555094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6e02-24e9-4531-b9db-466f03c25ce3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.582418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a742-0e16-453f-bfe1-917c24a0d930 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.589626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.590266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.590544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.590816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.592011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.593429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.595794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 523/1030] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:50.601258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.621423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.622331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.622804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.623077 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.630180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.631695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:50.646811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:50.647733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:50.648656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:50.649784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 508/1031] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:50.654143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.659802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.714931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b871-389d-40af-86b2-81bac7f88eb3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.720829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.753334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.754137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.754425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.754700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.755376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.756954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.759965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313ff60-0639-41a4-ae18-e6ce6d9f4533 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.799856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 524/1032] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:16:50 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:50.800095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.800095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.800095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.800095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.800447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:50.800447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.800447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee4cd4-4fbd-4dc0-b6b9-695a49d2e394 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.816020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 509/1033] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:50.820329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:50.934578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b53af-bd70-4715-9f28-45cd13b37530 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:50.937001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:50.977843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:50.978467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.978747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.979021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.979804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] RBAC: Adding query filter params (role.id=d4a05a85bf324fefae7801a6138c7d27, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:50.980142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:50.981565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:50.982557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320187-a71a-4bf1-aa48-c561641d6cac None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.982847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320187-a71a-4bf1-aa48-c561641d6cac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.983128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320187-a71a-4bf1-aa48-c561641d6cac None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:50.984782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 510/1034] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:50.987784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 525/1035] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:50 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=d4a05a85bf324fefae7801a6138c7d27 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:50.990113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:50.990394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:50.990669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.010580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:51.011814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:51.087730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 511/1036] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:51.110525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:51.214986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cc6a7-90a6-45c4-b974-3e951613fa67 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:51.248434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:51.249078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:51.249355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:51.249625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] PATH_INFO: `/v3/services/image` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.250238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:51.252178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:51.261042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:51.262099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:51.262959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:51.270330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 526/1037] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 15:16:51 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:51.270502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:51.325084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:51.369848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d70b5-21dd-4862-aa41-822bc7a41d54 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:51.403775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:51.404421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:51.404700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:51.404972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.405650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:51.405968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:51.407894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:51.417497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 512/1038] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:51.418568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:51.451886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:51.522465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3d91b-fb21-4ab6-8b3a-2ac28df4632b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:51.557372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:51.557993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:51.558275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:51.558550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.559233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:51.559515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:51.560909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:51.570628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 527/1039] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:51.573156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:51.677150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d292-1405-4dfc-9f51-9f05f815eaf9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:51.695316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:51.714226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:51.714846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:51.715126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:51.715398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.716178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] RBAC: Adding query filter params (interface=public, service_id=70b017f66d3147f08719a1deb6bcc4f6, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:51.716486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:51.717854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bd53f-cf31-46a6-81bd-5aab87eef1a6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:51.723010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 513/1040] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:16:51 2022] GET /identity/v3/endpoints?service_id=70b017f66d3147f08719a1deb6bcc4f6&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:51.728649 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:51.828914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:51.836305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0708e4-b933-4e1f-a484-be31b702d8ce None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:51.870889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:51.871503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:51.871782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:51.872072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:51.872823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:51.874167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cef5d-4b55-47b0-8bf6-51819a9a7734 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:51.886928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 528/1041] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:16:51 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:16:52.873905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee184-d4b7-437f-8fc8-2de57ec662f7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:52.874432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee184-d4b7-437f-8fc8-2de57ec662f7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:52.874510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee184-d4b7-437f-8fc8-2de57ec662f7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:52.875912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 514/1042] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:52.881010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:52.881430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:52.881573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:52.900342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:52.901521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:52.965015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f669-12e8-4694-b836-f1c043ff1988 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:52.965715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f669-12e8-4694-b836-f1c043ff1988 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:52.965715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f669-12e8-4694-b836-f1c043ff1988 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:52.966926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 515/1043] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:52.972537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:52.972924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:52.973100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:52.980542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 529/1044] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:52.992370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:52.993569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:53.004936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.067873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 516/1045] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:53.087551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.102346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:53.113384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f0763-a9a2-4dce-b5ba-dd2a96016adb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.148234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.148804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.149081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.149362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.150525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.152003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.153621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 530/1046] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:53.159324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.198008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cea7b-88f4-4232-b719-cb1290c8de29 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.212345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:53.235125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.236021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.236444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.236859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.247466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:53.253371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:53.254460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:53.255495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:53.257712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 517/1047] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:53.263590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75eadc-d4cf-47fe-a5e5-6eff70a1e21f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.263879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.300539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.301177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.301458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.301730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.302434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.303862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.327364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 531/1048] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:16:53 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:53.376189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1a561-549f-4754-a638-48d0d341978d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.413564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.414109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.414393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.414669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.415402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:53.415707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.417145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.431307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 518/1049] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:53.432658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.451714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:53.552125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5465-3fcf-48c9-b9bb-5f2fccf5288b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.589377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.590039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.590327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.590608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.601706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.603156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:53.618199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:53.619114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:53.620583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:53.620583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:53.620583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:53.620583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:53.620583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 532/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:53.625367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.661840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:53.730765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632bf65-3adc-49a2-b788-a88e0b23fb61 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.764681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.765306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.765590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.765868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.766898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:53.767205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.768610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41fe59-f0df-4feb-9a35-43775c191f5b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.785680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 519/1051] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:53.787483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:53.857413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:53.892923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d733-56f5-462b-aae6-815f6166a6d3 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.893440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d733-56f5-462b-aae6-815f6166a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.893440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820d733-56f5-462b-aae6-815f6166a6d3 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.894215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466f1d-ffb2-46cd-90e0-663a7436d7b2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:53.894931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 520/1052] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:53.899494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.899780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.900089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.919188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:53.920385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:53.931071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:53.931709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:53.932023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:53.932307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:53.939954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:53.941388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:53.955784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:53.956781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:53.957720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 533/1053] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:53.962783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.003013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 521/1054] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:54.025071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.052745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.071447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0de0e2-2a31-4bf7-af28-d7031765eed1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.102376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.111077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.111695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.112010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.112286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.113075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:54.113380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.114782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.133766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 534/1055] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:54.138828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.143311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49d638-7a3f-4204-b328-2d2424edf954 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.179600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.180268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.180544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.180819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] PATH_INFO: `/v3/services/image` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.181441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.182809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:54.192945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:54.193957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:54.194801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 522/1056] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:54.200369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.247343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb365-21d9-4bbf-9a63-655e285c2791 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.282712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.283276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.283556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.283833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.293822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.295266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.295687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:54.310261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:54.311173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:54.312133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:54.312133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:54.312133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:54.312133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:54.312133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 535/1057] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:54.313256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e04f6-4bfb-4b7e-b4a3-496659ad6697 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.316936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.350961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.351684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.352072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.352517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.352999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.353694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] RBAC: Adding query filter params (name=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:54.353999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.355357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.366425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 523/1058] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:54.367314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.421650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554cdcc-f4b3-4e9a-9818-3749604e74ed None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.458503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.459133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.459423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.459965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.460396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:54.460703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.462074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.479144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f6d7-c55a-4c1e-9edf-3bb287142688 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.479354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 536/1059] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:54.480364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.515401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.516611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.516902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.517178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.517847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:54.518145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.519506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.529458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 524/1060] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:54.530220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.548748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.590749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.597672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124fdc-b5c0-4af4-871b-bf62e96d6594 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.636188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dfb7e-53c4-4d7e-8bd4-d04a16be1fa1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.638754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.639397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.639679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.639976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.647625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.649229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:54.661446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:54.662312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:54.663201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 537/1061] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:54.668179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.672679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.673295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.673577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.673867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.674566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.675968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.699988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 525/1062] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:16:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:16:54.788687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924765d-dae3-4bc4-9911-e2a05bbf2bcc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:54.832671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:54.833214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:54.833489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:54.833756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:54.834512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:54.834817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:54.836297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff84467-4fc5-4d81-bc01-a2263aa60eab None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:54.854395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 538/1063] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:54.855851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:54.925701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:54.967097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab00c-a936-4277-9b0f-487c5efe79b9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:55.001861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:55.002525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:55.002802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:55.003070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/d4a05a85bf324fefae7801a6138c7d27` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:55.005248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=fe58a99358bc42aebeb126e5ef2cf21a, role_id=d4a05a85bf324fefae7801a6138c7d27)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:55.006685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcffc8-52e2-4357-abcb-64eadbe566f9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:55.021691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 526/1064] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:54 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/fe58a99358bc42aebeb126e5ef2cf21a/roles/d4a05a85bf324fefae7801a6138c7d27 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:16:55.417187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983beb-db8d-4df1-a236-1ab34cd20d2e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:55.417744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983beb-db8d-4df1-a236-1ab34cd20d2e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:55.417744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983beb-db8d-4df1-a236-1ab34cd20d2e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:55.419136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 539/1065] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:55.424478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:55.424835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:55.425200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:55.444167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:55.445344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:55.514086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 527/1066] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:55.537041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df983beb-db8d-4df1-a236-1ab34cd20d2e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:55.552678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:55.637338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df983beb-db8d-4df1-a236-1ab34cd20d2e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:55.669506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:55.670052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:55.670329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:55.670601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:55.681424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:55.686624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:55.687592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:55.689106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:55.690488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 540/1067] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:55.694501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:55.759398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:55.799437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075d13-0880-41eb-a286-3931e1ff8915 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:55.833934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:55.834547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:55.834824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:55.835093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:55.835800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:55.836129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:55.837523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:55.850800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 528/1068] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:55.852013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:55.928549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:55.966047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0ed7-2fc2-4420-8326-44735e0cc748 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:56.001344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:56.001989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.002270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.002543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] PATH_INFO: `/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.015945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] RBAC: Authorizing `identity:get_user(user_id=06f905ab8f9c421a8b1f05d2df91a1a8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:56.017360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:56.023730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 541/1069] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:16:55 2022] GET /identity/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:56.024689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:56.131168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc719e2-7d48-49b9-854c-ed1b33a8db50 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:56.154784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:56.167175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:56.167794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.168093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.168363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.238265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:56.239641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:56.339873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 529/1070] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:56.342521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:56.440776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646a64-ba8c-4d4f-9ff8-500933c8a16a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:56.470901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:56.478052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:56.478827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.479958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.479958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.480997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:56.482412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002a4e5-2445-417a-98c2-8093c0e2afc9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:56.489680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 542/1071] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:16:56 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:56.490665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:56.516316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:56.592387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d683e2-9bc2-42f1-bf82-cd88fbb95cef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:56.626792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:56.627405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.627682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.627974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.628759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=be0a038da3314e21a8a65aafb7d27825, user.id=06f905ab8f9c421a8b1f05d2df91a1a8) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:56.629063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:56.630489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db199f78-90e6-4526-a3b1-4e13445c5096 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:56.644110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 530/1072] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:56 2022] GET /identity/v3/role_assignments?user.id=06f905ab8f9c421a8b1f05d2df91a1a8&scope.project.id=be0a038da3314e21a8a65aafb7d27825&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:56.769000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e046-51e0-4d95-a04e-66a6faae6197 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.769305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e046-51e0-4d95-a04e-66a6faae6197 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.769474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e046-51e0-4d95-a04e-66a6faae6197 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.771038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 543/1073] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:16:56.776484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55de6-5af8-4eea-a832-d6e90e1bc0e4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:56.776815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55de6-5af8-4eea-a832-d6e90e1bc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:56.777054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55de6-5af8-4eea-a832-d6e90e1bc0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:56.795653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-01e55de6-5af8-4eea-a832-d6e90e1bc0e4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:56.796893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-01e55de6-5af8-4eea-a832-d6e90e1bc0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:56.864965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 531/1074] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:16:56.886897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e046-51e0-4d95-a04e-66a6faae6197 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:56.995771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e046-51e0-4d95-a04e-66a6faae6197 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.003270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4613-41f2-4c89-9c9b-96d32007f05e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.003561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4613-41f2-4c89-9c9b-96d32007f05e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.004133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4613-41f2-4c89-9c9b-96d32007f05e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.007939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 532/1075] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:57.010243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.010533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.011110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.029786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:57.030989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:57.031095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.032257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.032532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.032801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.046431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.104414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 533/1076] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:57.112043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.112043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.112043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.127321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.224189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 544/1077] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:57.229640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.230301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41753ee-e456-4c9e-a143-7b055195d537 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.267507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.268198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.268481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.268757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.279918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:57.285361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:57.286330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:57.287927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 534/1078] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:57.292581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.304253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.336602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.341397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc50b7-d11f-4564-a5e6-5ad76507c880 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.377190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.377832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.378115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.378394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.388932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.390364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.402948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d686a-bdea-4cf8-8349-3cac0e6c89a4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:57.405696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:57.407274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:57.408376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:57.408376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:57.408376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:57.408376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:57.408376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 545/1079] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:57.413278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.442301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.442914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.443233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.443456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.444187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:57.444491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.445856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.454643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 535/1080] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:57.460123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.517308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf0563-4222-481e-9571-1513ff122f76 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.531687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.554373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.555233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.555516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.555787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.556500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:57.556803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.558157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.566368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e4ee3-7ab6-403d-9fa6-e7c8e2924b65 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.570915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 546/1081] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:57.605048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.605767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.606049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.606322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.619093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.620621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:57.636723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:57.637734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:57.638632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:57.638632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:57.638632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:57.638632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:57.640196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 536/1082] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:57.643822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.658278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.768218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887630b7-7dfd-4ef5-a2f8-2f14c4fb9103 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.805446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.805987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.806268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.806543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.807233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:57.807535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.808928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.821229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 547/1083] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:57.826890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:57.938026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc711cde-23ce-4a5a-aa94-d84e77c8a000 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:57.960257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:57.973210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:57.973847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:57.974127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:57.974400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:57.981779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:57.983227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:16:57.994303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:57.995284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 15:16:57.996356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:57.999972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 537/1084] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 15:16:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:16:58.001290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.068858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:58.101940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebdd30-c668-4b7c-88b7-533a1cef0fa3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.134139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.134858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.135139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.135406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.136214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:58.136518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:58.137920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:58.155050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 548/1085] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:58.161327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.259757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7937ca-ca84-4219-8bd0-e7960aba66c4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.292941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.293554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.293835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.294110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.343043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:58.361088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:58.362548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:58.457485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 538/1086] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:16:58.463189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.559314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e15cd-41bd-4ceb-b1b5-eedc7628d148 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.585360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:58.593301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.593896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.594156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.594430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.604350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:58.605775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-708ba143-b3c5-4732-81fd-a89cece4749a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:16:58.620711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:58.621605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:58.622498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:16:58.622498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:16:58.622498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:16:58.622498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:58.624123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 549/1087] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:58.628193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.630580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777a0b-a98f-4a64-9066-78dfd12f6a9c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.630867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777a0b-a98f-4a64-9066-78dfd12f6a9c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.631146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777a0b-a98f-4a64-9066-78dfd12f6a9c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.632467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 550/1088] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:16:58.637383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.637666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.637940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.655351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:16:58.656576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:16:58.708661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:58.724212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 551/1089] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:16:58.727016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab899-c02c-4f94-b18d-107f1c55f94f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.746793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.762570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.763187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.763469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.763742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.764461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:58.764766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:58.766163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:58.776947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 539/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:16:58.782393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.848581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e3cc8-b846-4b0c-b55b-5a620ae618b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.859990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:58.882316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83965-2e62-4df9-8682-31fddc007596 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:58.884589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.885232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.885517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.885794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.896285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:16:58.901266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:58.902395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:16:58.903437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 552/1091] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:58.908933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.918687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:58.919318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:58.919600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:58.919903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:58.927371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:58.928863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:16:58.940209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:16:58.941091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:16:58.942025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 540/1092] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:16:58.947346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:58.950125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:59.010121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47fde-6bad-4b04-a5b2-aa0887b208ae None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.042528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.043139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.043417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.043690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.044422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:59.044726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.045581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac51da-5a80-4a15-af60-85a1a61da6e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.046083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.058360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 553/1093] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:59.064136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:59.079230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.079865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.080149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.080420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.081162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:59.081462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.082874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.094904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 541/1094] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:59.095208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:59.101348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:59.160648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:59.162881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f877483-5ed9-474f-9473-84754ba8f563 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.197677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.198307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.198590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.198872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] PATH_INFO: `/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] RBAC: Authorizing `identity:get_user(user_id=06f905ab8f9c421a8b1f05d2df91a1a8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.201387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798deeb8-20cd-4c82-be14-6d382eda29fe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.201822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.203222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 554/1095] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:16:59 2022] GET /identity/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:59.208645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:59.237635 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.238254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.238533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.238807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.239596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] RBAC: Adding query filter params (role.id=d4a05a85bf324fefae7801a6138c7d27, scope.project.id=3a947acc76624771ba3560291d433410, user.id=fe58a99358bc42aebeb126e5ef2cf21a) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:16:59.239926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.241354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.247011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 542/1096] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:16:59 2022] GET /identity/v3/role_assignments?user.id=fe58a99358bc42aebeb126e5ef2cf21a&scope.project.id=3a947acc76624771ba3560291d433410&role.id=d4a05a85bf324fefae7801a6138c7d27 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:59.310675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:59.314289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c10cbf-8f13-44e6-ad82-f0a86d878026 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.351628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839131-b011-4606-962e-430147a92435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.352232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839131-b011-4606-962e-430147a92435 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.352519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839131-b011-4606-962e-430147a92435 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.352797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839131-b011-4606-962e-430147a92435 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.354004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839131-b011-4606-962e-430147a92435 None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.355462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839131-b011-4606-962e-430147a92435 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.357416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 555/1097] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:16:59 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:16:59.363531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:16:59.464372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36992e-cddc-4948-ab6d-80098a329297 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:16:59.490238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:16:59.499457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:16:59.500230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:16:59.500510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:16:59.500780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825/users/06f905ab8f9c421a8b1f05d2df91a1a8/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:16:59.502699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] RBAC: Authorizing `identity:create_grant(project_id=be0a038da3314e21a8a65aafb7d27825, user_id=06f905ab8f9c421a8b1f05d2df91a1a8, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:16:59.504177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6869f-1849-4573-9204-be3f5a57e823 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:16:59.517446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 543/1098] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:16:59 2022] PUT /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825/users/06f905ab8f9c421a8b1f05d2df91a1a8/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:17:01.512013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76562c9-821c-48a0-943f-2fb92da996c4 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:01.512502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76562c9-821c-48a0-943f-2fb92da996c4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:01.512589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76562c9-821c-48a0-943f-2fb92da996c4 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:01.513914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 556/1099] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:01.519781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:01.520218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:01.520339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:01.537759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:01.538924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:01.603691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 544/1100] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:01.627684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76562c9-821c-48a0-943f-2fb92da996c4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:01.727253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76562c9-821c-48a0-943f-2fb92da996c4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:01.760481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:01.761102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:01.761382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:01.761653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:01.771507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:01.776396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:01.777459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:01.778493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 557/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:01.783580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:01.880207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318909e-49b7-4fa5-84b5-7fe76c91e621 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:01.899007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:01.918009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:01.918617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:01.918893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:01.919163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:01.919893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:01.920195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:01.921556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:01.929923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 545/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:01.935698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:01.964091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:02.033374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3c448-1155-46c1-9c73-c8fca51827b7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:02.065526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:02.066148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:02.066427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:02.066700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] PATH_INFO: `/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:02.067856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] RBAC: Authorizing `identity:get_user(user_id=06f905ab8f9c421a8b1f05d2df91a1a8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:02.069264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:02.070823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 558/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:01 2022] GET /identity/v3/users/06f905ab8f9c421a8b1f05d2df91a1a8 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:02.076800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:02.171421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a570-ad82-417b-b882-938110e07fb2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:02.203345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:02.203970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:02.204249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:02.204518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:02.247716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:02.269086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:02.270611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:02.366009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 546/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:02.371909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:02.463109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:02.469743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f25d0-dbac-407a-8a00-f76bd37caee4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:02.501890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:02.502512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:02.502794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:02.503068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:02.504232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:02.505644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397c4b2-8094-4bfa-8fa4-7b79cab1a098 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:02.507260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 559/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:02 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:02.513248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:02.587874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:02.608730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99897aa-aeca-46b3-8577-92bc7a44549a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:02.640927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:02.641485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:02.641761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:02.642030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:02.642804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=be0a038da3314e21a8a65aafb7d27825, user.id=06f905ab8f9c421a8b1f05d2df91a1a8) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:02.643106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:02.644549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a90eb-c504-47a5-952b-b6ff6c09c19b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:02.650072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 547/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:02 2022] GET /identity/v3/role_assignments?user.id=06f905ab8f9c421a8b1f05d2df91a1a8&scope.project.id=be0a038da3314e21a8a65aafb7d27825&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:04.661111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6075-01be-4801-a766-c53e2147298c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:04.661636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6075-01be-4801-a766-c53e2147298c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:04.661636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6075-01be-4801-a766-c53e2147298c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:04.663020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 560/1107] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:04.668369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:04.668778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:04.668929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:04.686498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:04.687716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:04.752935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 548/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:04.777273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a6075-01be-4801-a766-c53e2147298c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:04.875543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a6075-01be-4801-a766-c53e2147298c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:04.909600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:04.910225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:04.910506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:04.910779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:04.911991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:04.913374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51add746-57e6-4fb2-b6a4-2cf29d9200ae None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:04.914948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 561/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:17:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:04.920722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:04.987888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:05.017081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc66f4-5351-4362-b83d-a1ea426c6ed9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:05.049306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:05.049916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:05.050200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:05.050475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:05.051155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:05.052620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc82849-3663-4ba7-962e-e79944d52eb8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:05.073380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 549/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:17:04 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:17:07.114358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd4fc-d846-4752-a976-942c6ef3042e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.114865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd4fc-d846-4752-a976-942c6ef3042e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.114982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd4fc-d846-4752-a976-942c6ef3042e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.116289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 562/1111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:07.121397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.121791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.121954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.140232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:07.141421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:07.205532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 550/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:07.229096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbd4fc-d846-4752-a976-942c6ef3042e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:07.259243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:07.334082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbd4fc-d846-4752-a976-942c6ef3042e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:07.366901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:07.367528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.367807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.368120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] PATH_INFO: `/v3/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.369527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] RBAC: Authorizing `identity:get_role(role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:07.370868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:07.372217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 563/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:07 2022] GET /identity/v3/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:07.378172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:07.434088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:07.474657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8a47d-af31-4145-be71-5e3b07ad433f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:07.507444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:07.508048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.508333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.508609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] PATH_INFO: `/v3/users/63a4d79e8bc34d1eb57a3a762437943e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.521074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] RBAC: Authorizing `identity:get_user(user_id=63a4d79e8bc34d1eb57a3a762437943e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:07.522514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:07.524153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 551/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:07 2022] GET /identity/v3/users/63a4d79e8bc34d1eb57a3a762437943e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:07.529920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:07.623261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:07.625941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e370-853a-4f35-8818-ea28929ee9e3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:07.659278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:07.659899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.660181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.660452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.727145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:07.728607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:07.819602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:07.824437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 564/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:07.829747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:07.932572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be003e32-f89d-4e3f-a949-ec9607a271dd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:07.950157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:07.971316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:07.971991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:07.972277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:07.972555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:07.973706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:07.975138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7a5ef-738d-44ac-9ce3-cbb688a9448b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:07.976820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 552/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:07 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:07.982896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:08.082320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7def50-7673-45dd-b06b-e29b8aa607fd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:08.115152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:08.115745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:08.116046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:08.116316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:08.117085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=be0a038da3314e21a8a65aafb7d27825, user.id=63a4d79e8bc34d1eb57a3a762437943e) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:08.117393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:08.118788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295368a4-e744-4753-8d86-13673cf9b6c9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:08.123969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 565/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:07 2022] GET /identity/v3/role_assignments?user.id=63a4d79e8bc34d1eb57a3a762437943e&scope.project.id=be0a038da3314e21a8a65aafb7d27825&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:10.099152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d313-6ccc-4566-a105-d1197eb92a7a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.099666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d313-6ccc-4566-a105-d1197eb92a7a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.099761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d313-6ccc-4566-a105-d1197eb92a7a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.101046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 553/1118] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:10.106313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.106684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.106883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.124421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:10.125581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:10.181493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:10.190668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 566/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:10.213301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d313-6ccc-4566-a105-d1197eb92a7a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:10.317122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d313-6ccc-4566-a105-d1197eb92a7a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:10.349387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:10.350014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.350292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.350562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] PATH_INFO: `/v3/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.351997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] RBAC: Authorizing `identity:get_role(role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:10.353369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:10.354679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 554/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:10 2022] GET /identity/v3/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:10.360707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:10.456758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80d0ac-787e-44fb-b9ae-82719ccd2470 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:10.488468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:10.489103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.489386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.489664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] PATH_INFO: `/v3/users/63a4d79e8bc34d1eb57a3a762437943e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.490822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] RBAC: Authorizing `identity:get_user(user_id=63a4d79e8bc34d1eb57a3a762437943e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:10.492268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:10.493835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 567/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:10 2022] GET /identity/v3/users/63a4d79e8bc34d1eb57a3a762437943e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:10.499974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:10.550645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:10.600594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8232a3-bc38-498f-85e9-e8970699e418 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:10.633958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:10.634584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.634865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.635140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.636305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:10.637741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550611b6-2579-4053-8295-2ab58d56e725 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:10.639378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 555/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:10 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:10.645236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:10.705262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:10.742290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb136b00-e179-47ce-9b1a-aebeb176c6e2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:10.775167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:10.775817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:10.776127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:10.776398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825/users/63a4d79e8bc34d1eb57a3a762437943e/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:10.778519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=be0a038da3314e21a8a65aafb7d27825, user_id=63a4d79e8bc34d1eb57a3a762437943e, role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:10.779946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27effb-b11a-4ebf-9275-f40597795a0c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:10.791817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 568/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:17:10 2022] PUT /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825/users/63a4d79e8bc34d1eb57a3a762437943e/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:17:12.743425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650591c-ac72-45b4-b473-3b675439c5ca None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:12.744006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650591c-ac72-45b4-b473-3b675439c5ca None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:12.744006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650591c-ac72-45b4-b473-3b675439c5ca None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:12.745342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 556/1124] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:12.750986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:12.751412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:12.751520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:12.769875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:12.771045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:12.835276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 569/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:12.857891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650591c-ac72-45b4-b473-3b675439c5ca None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:12.958964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650591c-ac72-45b4-b473-3b675439c5ca None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:12.978983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:12.992872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:12.993551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:12.993785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:12.994059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] PATH_INFO: `/v3/roles/7bc94b9d8c8743c89ca9805057a60cad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:12.995479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] RBAC: Authorizing `identity:get_role(role_id=7bc94b9d8c8743c89ca9805057a60cad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:12.996879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:12.998205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 557/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:12 2022] GET /identity/v3/roles/7bc94b9d8c8743c89ca9805057a60cad => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:13.004023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:13.054960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:13.101355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2037-be7c-4a26-99d8-1b1caa4c40a7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:13.133413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:13.134036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:13.134318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:13.134593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] PATH_INFO: `/v3/users/63a4d79e8bc34d1eb57a3a762437943e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:13.135735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] RBAC: Authorizing `identity:get_user(user_id=63a4d79e8bc34d1eb57a3a762437943e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:13.137174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:13.138752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 570/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:13 2022] GET /identity/v3/users/63a4d79e8bc34d1eb57a3a762437943e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:13.144945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:13.241049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1cacd-d982-4921-9426-b5c5d65a26a0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:13.274613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:13.275148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:13.275426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:13.275697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:13.328536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:13.341141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:13.342595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:13.436244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 558/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:13.442591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:13.539260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18948cc3-ece8-4576-8236-c9e0ad672fba None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:13.559495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:13.574507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:13.575142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:13.575423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:13.575697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] PATH_INFO: `/v3/projects/be0a038da3314e21a8a65aafb7d27825` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:13.576966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] RBAC: Authorizing `identity:get_project(project_id=be0a038da3314e21a8a65aafb7d27825)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:13.578345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460227e-f8b9-4574-bded-eff8ec54d608 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:13.580033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 571/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:13 2022] GET /identity/v3/projects/be0a038da3314e21a8a65aafb7d27825 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:13.586246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:13.670037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:13.682311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993fa6-cc41-4742-9c80-e01c45877dc3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:13.715211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:13.715816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:13.716128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:13.716407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:13.717190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] RBAC: Adding query filter params (role.id=7bc94b9d8c8743c89ca9805057a60cad, scope.project.id=be0a038da3314e21a8a65aafb7d27825, user.id=63a4d79e8bc34d1eb57a3a762437943e) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:13.717502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:13.718970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b983c-edde-4cbf-bec2-6d563ea02178 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:13.724605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 559/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:13 2022] GET /identity/v3/role_assignments?user.id=63a4d79e8bc34d1eb57a3a762437943e&scope.project.id=be0a038da3314e21a8a65aafb7d27825&role.id=7bc94b9d8c8743c89ca9805057a60cad => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:15.728358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559b84-b6f4-4ac9-bf07-737529106cbf None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:15.728907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559b84-b6f4-4ac9-bf07-737529106cbf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:15.728907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559b84-b6f4-4ac9-bf07-737529106cbf None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:15.730249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 572/1131] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:15.736080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:15.736485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:15.736629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:15.754659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:15.755823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:15.820866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 560/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:15.843428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b559b84-b6f4-4ac9-bf07-737529106cbf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:15.945413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b559b84-b6f4-4ac9-bf07-737529106cbf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:15.984404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:15.984987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:15.985265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:15.985536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:15.986679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:15.988135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f06990-970b-4b0b-b2d6-dfc09b5eb807 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:15.989721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 573/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:17:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:15.995523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:16.073191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:16.090680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9a046-7535-4618-b5b7-af0a33ab9080 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:16.123011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:16.123618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:16.123983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:16.124761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:16.125443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:16.126869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb314a31-2ba5-4944-8b2d-c979bd201751 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:16.151880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 561/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:17:15 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:17:18.154749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca0ec-ea4c-4bb4-b4ce-ac48e6c52c87 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:18.155818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca0ec-ea4c-4bb4-b4ce-ac48e6c52c87 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:18.155944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca0ec-ea4c-4bb4-b4ce-ac48e6c52c87 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:18.157261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 574/1135] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:18.162308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:18.162735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:18.162838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:18.180450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:18.181612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:18.245180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 562/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:18.269530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ca0ec-ea4c-4bb4-b4ce-ac48e6c52c87 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:18.325597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:18.369656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ca0ec-ea4c-4bb4-b4ce-ac48e6c52c87 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:18.402682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:18.403311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:18.403594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:18.403897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:18.405628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:18.407057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7cdee-d5f5-4757-802c-a18784b6025a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:18.408667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 575/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:18.414359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:18.473400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:18.510223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5a59-01c1-47ea-9d38-82cfaf64bca5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:18.542436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:18.543035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:18.543311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:18.543583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:18.544284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:18.545736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54c981-79ea-4513-99f9-312ead57ae35 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:18.566235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 563/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:17:18 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:17:20.558507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8007-3504-43e4-b13d-190ab4b4ddd1 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:20.559062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8007-3504-43e4-b13d-190ab4b4ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:20.559062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8007-3504-43e4-b13d-190ab4b4ddd1 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:20.560443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 576/1139] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:20.566228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:20.566692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:20.566790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:20.585126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:20.586316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:20.651206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 564/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:20.674962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8007-3504-43e4-b13d-190ab4b4ddd1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:20.772713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8007-3504-43e4-b13d-190ab4b4ddd1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:20.795409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:20.806468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:20.807092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:20.807372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:20.807644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:20.817547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:20.822681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:20.823856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:20.824975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 577/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:20.829894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:20.878252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:20.926961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911140b-bc84-4c76-9766-e0222ffe6262 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:20.959457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:20.960128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:20.960412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:20.960692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:20.961406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:20.961713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:20.963093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:20.971620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 565/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:20.977409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:21.074313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4729855-9c67-4785-b648-a82f5f3de9d5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:21.107342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:21.108008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:21.108292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:21.108566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] PATH_INFO: `/v3/users/8bafce58f0994e1981ca6e0506ec7f60` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:21.121127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] RBAC: Authorizing `identity:get_user(user_id=8bafce58f0994e1981ca6e0506ec7f60)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:21.122552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:21.124177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 578/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:20 2022] GET /identity/v3/users/8bafce58f0994e1981ca6e0506ec7f60 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:21.129755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:21.227030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d7d26-d19c-4683-a70f-9d40db77f4e8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:21.239232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:21.260256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:21.260862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:21.261145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:21.261422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:21.325826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:21.327760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:21.422542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 566/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:21.428460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:21.470644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:21.525137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845fbee-4781-4fc3-9258-22063a1ea5fa None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:21.558872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:21.559489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:21.559767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:21.560065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] PATH_INFO: `/v3/projects/1e99ab6b5f274292be902f5bab278fc0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:21.561217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] RBAC: Authorizing `identity:get_project(project_id=1e99ab6b5f274292be902f5bab278fc0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:21.562631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e573dab-6332-4a3a-a8fe-78340a3f1437 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:21.564281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 579/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:21 2022] GET /identity/v3/projects/1e99ab6b5f274292be902f5bab278fc0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:21.570184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:21.584392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:21.668519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcbcdd-4786-44b5-b7e4-146a248b02fc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:21.701309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:21.701950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:21.702231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:21.702504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:21.703288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=1e99ab6b5f274292be902f5bab278fc0, user.id=8bafce58f0994e1981ca6e0506ec7f60) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:21.703597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:21.705050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4976-32e2-47fd-8e3b-dceafaceaa2f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:21.710249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 567/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:21 2022] GET /identity/v3/role_assignments?user.id=8bafce58f0994e1981ca6e0506ec7f60&scope.project.id=1e99ab6b5f274292be902f5bab278fc0&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:23.650141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9987f-967a-443c-aecb-ea5ca48db5b7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:23.650670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9987f-967a-443c-aecb-ea5ca48db5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:23.650670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9987f-967a-443c-aecb-ea5ca48db5b7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:23.652070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 580/1147] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:23.657595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:23.658006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:23.658158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:23.676431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:23.677612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:23.741969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 568/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:23.763414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9987f-967a-443c-aecb-ea5ca48db5b7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:23.862406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9987f-967a-443c-aecb-ea5ca48db5b7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:23.872669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:23.895204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:23.895845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:23.896134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:23.896409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:23.906800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:23.911688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:23.912803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:23.913690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 581/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:23.918783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:23.927281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:24.016165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003302-ed2c-443b-84ed-c07e927db47e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:24.055122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:24.055742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:24.056053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:24.056325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:24.057029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:24.057333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:24.058717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:24.066782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 569/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:24.072872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:24.169905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea09e-2a9f-4f27-9f09-a150d65f3725 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:24.202605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:24.203221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:24.203502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:24.203775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] PATH_INFO: `/v3/users/8bafce58f0994e1981ca6e0506ec7f60` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:24.204946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] RBAC: Authorizing `identity:get_user(user_id=8bafce58f0994e1981ca6e0506ec7f60)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:24.206359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:24.207946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 582/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:24 2022] GET /identity/v3/users/8bafce58f0994e1981ca6e0506ec7f60 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:24.213445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:24.276754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:24.309291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd8bf-f2c9-480e-8413-fb210b84557a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:24.342736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:24.343365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:24.343640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:24.343937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] PATH_INFO: `/v3/projects/1e99ab6b5f274292be902f5bab278fc0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:24.345089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e99ab6b5f274292be902f5bab278fc0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:24.346516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebbf5d-ee82-4c7a-8331-0aa8693303b7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:24.348365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 570/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:24 2022] GET /identity/v3/projects/1e99ab6b5f274292be902f5bab278fc0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:24.354450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:24.393577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:24.452417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35585-2dfd-435c-9620-cbc3f5419c20 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:24.484116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:24.484764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:24.485043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:24.485314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] PATH_INFO: `/v3/projects/1e99ab6b5f274292be902f5bab278fc0/users/8bafce58f0994e1981ca6e0506ec7f60/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:24.487187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] RBAC: Authorizing `identity:create_grant(project_id=1e99ab6b5f274292be902f5bab278fc0, user_id=8bafce58f0994e1981ca6e0506ec7f60, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:24.488620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fab28-43ee-40e0-bf1e-5ad97adad34b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:24.502035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 583/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:17:24 2022] PUT /identity/v3/projects/1e99ab6b5f274292be902f5bab278fc0/users/8bafce58f0994e1981ca6e0506ec7f60/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:17:26.459939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420eb2a-9928-41e5-8cf6-0112fb42e5de None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:26.460458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420eb2a-9928-41e5-8cf6-0112fb42e5de None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:26.460549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420eb2a-9928-41e5-8cf6-0112fb42e5de None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:26.461814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 571/1154] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:26.467076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:26.467492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:26.467621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:26.485676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:26.486831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:26.549853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 584/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:26.573071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420eb2a-9928-41e5-8cf6-0112fb42e5de None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:26.671988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420eb2a-9928-41e5-8cf6-0112fb42e5de None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:26.703819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:26.704470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:26.704751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:26.705025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:26.707569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:26.716531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:26.722454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:26.723550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:17:26.724761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 572/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:26.729677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:26.756989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:26.827127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c8ce3-0887-437c-a02f-b794c80a10da None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:26.860318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:26.860923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:26.861205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:26.861479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:26.862183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:26.862489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:26.863860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:26.871625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 585/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:26.877299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:26.973082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9f3c5-f47e-441d-a619-d17e84d54fe3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:27.005563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:27.006232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:27.006510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:27.006784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] PATH_INFO: `/v3/users/8bafce58f0994e1981ca6e0506ec7f60` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:27.007954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] RBAC: Authorizing `identity:get_user(user_id=8bafce58f0994e1981ca6e0506ec7f60)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:27.009383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:27.010956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 573/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:26 2022] GET /identity/v3/users/8bafce58f0994e1981ca6e0506ec7f60 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:27.016824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:27.104782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:27.114357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376efdc4-e045-4118-97f9-a2dee6cbbd59 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:27.146214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:27.146805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:27.147083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:27.147353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:27.211071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:27.212514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:27.296257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:27.306472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 586/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 15:17:27.312386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:27.372119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:27.409237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cc792-a8a8-4368-9c46-bdab3daa48b2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:27.441026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:27.441646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:27.441929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:27.442205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] PATH_INFO: `/v3/projects/1e99ab6b5f274292be902f5bab278fc0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:27.443340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=1e99ab6b5f274292be902f5bab278fc0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:27.444797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780f2c0-249e-4052-9941-e6e4b1da0ac4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:27.446669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 574/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:27 2022] GET /identity/v3/projects/1e99ab6b5f274292be902f5bab278fc0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:27.453354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:27.547169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbb6a6-ccca-4c53-86a9-4737fe8161f3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:27.580800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:27.581401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:27.581684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:27.581959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:27.582736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=1e99ab6b5f274292be902f5bab278fc0, user.id=8bafce58f0994e1981ca6e0506ec7f60) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:27.583046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:27.584486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b6674-f3c9-4d8d-b6c7-50e4f8770505 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:27.590098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 587/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:27 2022] GET /identity/v3/role_assignments?user.id=8bafce58f0994e1981ca6e0506ec7f60&scope.project.id=1e99ab6b5f274292be902f5bab278fc0&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:29.566239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bf954-3cec-441d-93c4-1cac870605cb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:29.566798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bf954-3cec-441d-93c4-1cac870605cb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:29.566798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bf954-3cec-441d-93c4-1cac870605cb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:29.568143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 575/1162] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:29.573435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:29.573850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:29.573987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:29.592916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:29.594096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:29.641376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:29.658754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 588/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:29.680664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bf954-3cec-441d-93c4-1cac870605cb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:29.780313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bf954-3cec-441d-93c4-1cac870605cb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:29.807111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:29.812628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:29.813229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:29.813507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:29.813777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:29.879942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:29.881382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:29.975067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 576/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:29.981021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:30.084377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02482b39-2bbd-453c-b677-4bd3c4a94301 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:30.117312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:30.117943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:30.118226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:30.118501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:30.125692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:30.127119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a2645a30-6aac-425a-afed-0dead9b6594b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:30.138550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:30.139718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:30.140670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 589/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:17:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:30.145728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:30.175192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:30.241671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389e65e-7f3a-4454-a0dd-1ff045511616 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:30.274383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:30.274996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:30.275272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:30.275544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:30.276255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:30.276558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:30.277932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f95125-c5a4-4666-bf0d-229519e2babd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:30.284874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 577/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 15:17:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:32.288952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f57f-f012-4259-8cba-332d90fa1930 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:32.289462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f57f-f012-4259-8cba-332d90fa1930 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:32.289462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6f57f-f012-4259-8cba-332d90fa1930 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:32.290766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 590/1167] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:32.296004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8a93-c14e-4c87-87fa-392f6b7a911e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:32.296422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8a93-c14e-4c87-87fa-392f6b7a911e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:32.296559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8a93-c14e-4c87-87fa-392f6b7a911e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:32.314327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-828d8a93-c14e-4c87-87fa-392f6b7a911e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:32.315518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-828d8a93-c14e-4c87-87fa-392f6b7a911e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:32.380214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 578/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:32.402701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6f57f-f012-4259-8cba-332d90fa1930 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:32.449781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:32.502934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6f57f-f012-4259-8cba-332d90fa1930 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:32.536401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:32.537005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:32.537288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:32.537566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:32.538273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:32.539666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a81db-e003-4b38-b5a9-4ea27f69863d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:32.554651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 591/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Jul 27 15:17:32 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:17:34.534894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0a08-1f49-40a0-beee-6c3f7e095f98 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:34.535449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0a08-1f49-40a0-beee-6c3f7e095f98 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:34.535449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0a08-1f49-40a0-beee-6c3f7e095f98 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:34.536823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 579/1170] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:34.541808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:34.542178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:34.542381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:34.561037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:34.562201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:34.625566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 592/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:34.647437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c0a08-1f49-40a0-beee-6c3f7e095f98 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:34.674109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:34.745323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c0a08-1f49-40a0-beee-6c3f7e095f98 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:34.777695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:34.778316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:34.778596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:34.778871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] PATH_INFO: `/v3/domains/66412fa0619949a395a0acbc7b577fae` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:34.788974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=66412fa0619949a395a0acbc7b577fae)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:34.790374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbf00d-c700-41af-bf0e-efc639cc33a5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:34.791966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 580/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 15:17:34 2022] GET /identity/v3/domains/66412fa0619949a395a0acbc7b577fae => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:34.798202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:34.830815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:34.894533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b345f-65ac-41a2-85f2-1bda43e021d1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:34.927292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:34.927918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:34.928198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:34.928468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:34.929143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:34.930539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b30d14-2c78-4766-9a7c-33820b6fa7c3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:34.956168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 593/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:17:34 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:17:36.971514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732f9b-5866-4ab0-8670-39c12d5f2042 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:36.971975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732f9b-5866-4ab0-8670-39c12d5f2042 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:36.972116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732f9b-5866-4ab0-8670-39c12d5f2042 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:36.973474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 581/1174] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:36.978651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:36.979052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:36.979203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:36.997736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:36.999416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:37.063723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 594/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:37.087654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8732f9b-5866-4ab0-8670-39c12d5f2042 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:37.171530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:37.186856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8732f9b-5866-4ab0-8670-39c12d5f2042 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:37.219388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:37.219989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:37.220280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:37.220560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] PATH_INFO: `/v3/domains/66412fa0619949a395a0acbc7b577fae` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:37.221706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=66412fa0619949a395a0acbc7b577fae)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:37.223150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac2cfa-3863-43d1-a945-9b830cbd56a5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:37.224745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 582/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 15:17:37 2022] GET /identity/v3/domains/66412fa0619949a395a0acbc7b577fae => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:37.230371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:37.248154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:37.331906 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb8c97-da28-4448-91e4-572341a90199 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:37.365401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:37.366012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:37.366295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:37.366572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:37.367255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:37.368688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ca1d9-1bfb-4246-91df-436d46b70572 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:37.389165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 595/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:17:37 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:17:39.398665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c054-2e66-413e-aeb6-9bbd89901930 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:39.399186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c054-2e66-413e-aeb6-9bbd89901930 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:39.399303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c054-2e66-413e-aeb6-9bbd89901930 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:39.400602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 583/1178] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:39.407331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:39.407796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:39.407958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:39.426390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:39.427580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:39.492491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 596/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:17:39.514723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228c054-2e66-413e-aeb6-9bbd89901930 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:39.613096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228c054-2e66-413e-aeb6-9bbd89901930 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:39.644498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:39.645116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:39.645396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:39.645671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:39.656430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:39.661270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:39.662333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:17:39.663427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 584/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:39.670724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:39.672900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:39.769697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40c9a3-999e-4932-9825-9beab4ae6023 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:39.802212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:39.802819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:39.803099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:39.803374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:39.804097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:39.804401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:39.805746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:39.813515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 597/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:17:39.819298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:39.835567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:39.917182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf419-6838-42ff-9490-8264dc90120e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:39.950308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:39.951480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:39.951761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:39.952064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] PATH_INFO: `/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:39.965172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] RBAC: Authorizing `identity:get_user(user_id=31a2bfc7ba9d40afbbe76d5d2382f7a5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:39.967205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:39.968859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 585/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:39 2022] GET /identity/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:39.974527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:40.035448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:40.076498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a5f9-b9af-4b61-8eaa-b5c97bf0cc5b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:40.109053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:40.109656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:40.109937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:40.110211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:40.174493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:40.175873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:40.233348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:40.270914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 598/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:40.276939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:40.347899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:40.376094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cf4d5-a467-4b45-baf2-7a230c06235a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:40.408880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:40.409503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:40.409782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:40.410056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] PATH_INFO: `/v3/projects/10dc7469709b421f8db930a593054b9a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:40.411194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] RBAC: Authorizing `identity:get_project(project_id=10dc7469709b421f8db930a593054b9a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:40.412646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e3401-7a57-4697-97a9-0b8c64498d54 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:40.414259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 586/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:40 2022] GET /identity/v3/projects/10dc7469709b421f8db930a593054b9a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:40.420937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:40.516398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b0824-b40d-4257-ae74-bf6c626f7e2a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:40.548808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:40.549403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:40.549685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:40.549958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:40.550732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=10dc7469709b421f8db930a593054b9a, user.id=31a2bfc7ba9d40afbbe76d5d2382f7a5) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:40.551039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:40.552479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3b9b5-4795-458d-bb42-a4941d7490e3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:40.557669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 599/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:40 2022] GET /identity/v3/role_assignments?user.id=31a2bfc7ba9d40afbbe76d5d2382f7a5&scope.project.id=10dc7469709b421f8db930a593054b9a&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:42.528418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3dfa-464a-4809-90e1-89ab080b719f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:42.529523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3dfa-464a-4809-90e1-89ab080b719f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:42.529523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3dfa-464a-4809-90e1-89ab080b719f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:42.530823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 587/1186] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:42.536063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:42.536464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:42.536620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:42.555053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:42.556255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:42.579371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:42.621261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 600/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:42.644107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3dfa-464a-4809-90e1-89ab080b719f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:42.740447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3dfa-464a-4809-90e1-89ab080b719f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:42.773385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:42.774005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:42.774285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:42.774559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:42.781835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:42.785000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:42.790432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:42.791595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:17:42.792691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 588/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:42.797914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:42.893552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a1a71-03a1-4d91-90f0-0702ca7c293e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:42.926400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:42.926994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:42.927274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:42.927544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:42.928288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:42.928587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:42.929955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:42.938283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 601/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:42.944133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:43.039792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279e93-2649-49c0-9818-82de97cfde5f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:43.073350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:43.073981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:43.074261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:43.074537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] PATH_INFO: `/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:43.075713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] RBAC: Authorizing `identity:get_user(user_id=31a2bfc7ba9d40afbbe76d5d2382f7a5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:43.077180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:43.078733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 589/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:42 2022] GET /identity/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:43.084471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:43.106127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:43.183213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76ac1b-529a-4459-8d31-bb010b0b625f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:43.215364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:43.216544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:43.216831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:43.217108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] PATH_INFO: `/v3/projects/10dc7469709b421f8db930a593054b9a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:43.218252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] RBAC: Authorizing `identity:get_project(project_id=10dc7469709b421f8db930a593054b9a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:43.219659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcf258-90d2-4053-b54e-9f6a37b278bb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:43.221327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 602/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:43 2022] GET /identity/v3/projects/10dc7469709b421f8db930a593054b9a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:43.227233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:43.292263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:43.324731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35d2b-d280-461d-8ff0-fe3bfbd98e5a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:43.358049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:43.358692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:43.358972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:43.359247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] PATH_INFO: `/v3/projects/10dc7469709b421f8db930a593054b9a/users/31a2bfc7ba9d40afbbe76d5d2382f7a5/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:43.361183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] RBAC: Authorizing `identity:create_grant(project_id=10dc7469709b421f8db930a593054b9a, user_id=31a2bfc7ba9d40afbbe76d5d2382f7a5, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:43.362603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248518f-a77b-44fa-9b3a-b84fb6c708ea None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:43.374543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 590/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:17:43 2022] PUT /identity/v3/projects/10dc7469709b421f8db930a593054b9a/users/31a2bfc7ba9d40afbbe76d5d2382f7a5/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:17:45.344104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05ff3-59a1-4f7e-b999-e4c9a1f319e2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:45.344616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05ff3-59a1-4f7e-b999-e4c9a1f319e2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:45.344616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05ff3-59a1-4f7e-b999-e4c9a1f319e2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:45.345912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 603/1193] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:17:45.351092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:45.351519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:45.351624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:45.369191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:17:45.370364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:17:45.433941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 591/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:17:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:17:45.457401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05ff3-59a1-4f7e-b999-e4c9a1f319e2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:45.541907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:45.554418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05ff3-59a1-4f7e-b999-e4c9a1f319e2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:45.587065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:45.587681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:45.587993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:45.588265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:45.598501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:17:45.603343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:17:45.604464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:17:45.605381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 604/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:17:45.610347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:45.666875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:45.708100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236ddd5-3b20-4de1-8caa-fca5cfa8eb7c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:45.739992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:45.740597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:45.740876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:45.741146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:45.741847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:45.742145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:45.743501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:45.751639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 592/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:45.757419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:45.853346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f0cc7-c144-4eac-bc20-86d4c649cf6a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:45.885707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:45.886321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:45.886601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:45.886871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] PATH_INFO: `/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:45.888070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] RBAC: Authorizing `identity:get_user(user_id=31a2bfc7ba9d40afbbe76d5d2382f7a5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:45.889480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:45.891029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 605/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 15:17:45 2022] GET /identity/v3/users/31a2bfc7ba9d40afbbe76d5d2382f7a5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:45.896498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:45.994000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdeeb5-8ec6-44a0-9822-99a0ad6617e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:46.013203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:46.027031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:46.027638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:46.027950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:46.028225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:46.094231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:46.095679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:46.188877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 593/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:17:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:17:46.194668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:46.212818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:46.291443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b151f-2794-4d34-8fcd-de066634d679 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:46.323712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:46.324355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:46.324635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:46.324908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] PATH_INFO: `/v3/projects/10dc7469709b421f8db930a593054b9a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:46.326072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] RBAC: Authorizing `identity:get_project(project_id=10dc7469709b421f8db930a593054b9a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:46.327478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b1bee-2166-4b26-8ba0-f44140b538ab None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:46.329125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 606/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 15:17:46 2022] GET /identity/v3/projects/10dc7469709b421f8db930a593054b9a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:46.335244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:17:46.355711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:46.430483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f6422-a84a-41b3-a6a6-f2c048914783 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:17:46.463529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:17:46.464106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:17:46.464390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:17:46.464666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:17:46.465454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=10dc7469709b421f8db930a593054b9a, user.id=31a2bfc7ba9d40afbbe76d5d2382f7a5) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:17:46.466269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:17:46.467694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098776-921e-4ff5-85d1-919b72b4d241 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:17:46.473338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 594/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:17:46 2022] GET /identity/v3/role_assignments?user.id=31a2bfc7ba9d40afbbe76d5d2382f7a5&scope.project.id=10dc7469709b421f8db930a593054b9a&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:17:56.319793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=58 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:17:56.470793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=58 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:40.720919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322f83-f6fe-4f32-99ed-3e1bc72de8a7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:40.720919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322f83-f6fe-4f32-99ed-3e1bc72de8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:40.722136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322f83-f6fe-4f32-99ed-3e1bc72de8a7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:40.723801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 607/1201] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:18:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:18:40.729815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:40.730358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:40.730358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:40.749821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:18:40.751051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:18:40.845210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 595/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:18:40.875200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322f83-f6fe-4f32-99ed-3e1bc72de8a7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:41.017511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322f83-f6fe-4f32-99ed-3e1bc72de8a7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:41.070207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:41.070858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:41.071188 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:41.071475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:41.084849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:41.086760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:41.107908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:41.108961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:41.110513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:41.110513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:41.110513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:41.110513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:41.110513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 608/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:18:41.117506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:41.244508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb3cb4-8568-4070-a95e-c25104230391 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:41.284878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:41.292447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:41.293075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:41.293357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:41.293622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:41.294646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:41.295019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:41.296478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dab0-b300-4d68-b866-596c0d7b7651 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:41.309889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 596/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:41.316395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:41.384409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:41.443127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683763-779a-4037-a4ec-090cfbc1ff8f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:41.480566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:41.481192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:41.481550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:41.481745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:41.482420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:41.483896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241b572-323e-4b1d-9889-2a046500a47c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:41.520056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 609/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 15:18:41 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:18:43.801978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c56bd-7857-4fb8-a0e4-de3110719d69 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:43.802585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c56bd-7857-4fb8-a0e4-de3110719d69 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:43.802585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c56bd-7857-4fb8-a0e4-de3110719d69 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:43.803968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 597/1206] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:18:43.810121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:43.810669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:43.810669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:43.836775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:18:43.838552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:18:43.913042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 610/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:18:43.936901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c56bd-7857-4fb8-a0e4-de3110719d69 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:44.060258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c56bd-7857-4fb8-a0e4-de3110719d69 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:44.103303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:44.103979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:44.104261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:44.104533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:44.118158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:18:44.124600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:44.125651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:44.126554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 598/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:44.134366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:44.184376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:44.267191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eaf50-b14f-4dfe-aa47-4de6118f5a73 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:44.311849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:44.312472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:44.312875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:44.313196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:44.314145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:44.314598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:44.316387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:44.327723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 611/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:44.333372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:44.397258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:44.462321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ed44b-8191-4b7e-9c36-816c843a7d47 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:44.514077 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:44.514784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:44.515219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:44.515618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:44.529185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:44.530924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:44.548532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:44.549927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:44.551188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:44.551188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:44.551188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:44.551188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:44.554692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 599/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:44.555671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:44.680141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:44.684214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844654-edb2-4654-b8ff-3209a8bc89d5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:44.731578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:44.732645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:44.734805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:44.734805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:44.734805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:44.734805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:44.736836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:44.754522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 612/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:44.760943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:44.890824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26ad1-2c4b-4652-96ae-8d2a14bfbff8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:44.892801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:44.931380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:44.932107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:44.932404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:44.932783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:44.941082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:44.942940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:18:44.955422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:44.956708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:44.957685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 600/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:44.964734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:45.094488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835b7a-62d8-42c1-ab68-b0297c923771 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:45.138590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:45.139175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:45.139455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:45.139873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:45.141589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:45.141589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:45.142600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:45.162733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 613/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:18:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:45.168640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:45.290109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa2d5-60d4-4866-b710-0d0beaa9f518 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:45.330122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:45.330824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:45.331203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:45.331584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:45.365602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:45.408808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:45.410268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:45.526934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 601/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:18:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 15:18:45.532846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:45.557643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:45.653457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4f55-3c05-4511-b70f-ca90c9727604 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:45.693614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:45.694330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:45.694596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:45.694864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:45.709536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:45.711069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:45.730587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:45.731639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:45.732613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:45.732613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:45.732613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:45.732613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:45.732613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 614/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:18:45.738747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:45.817065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:45.860877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3333f9-1b2a-4782-8511-773d5add6c90 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:45.898569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:45.898569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:45.898569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:45.898569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:45.902385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:45.902385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:45.902385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:45.916045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 602/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:45.923266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:46.018612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:46.060494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ac04-033f-485b-aa14-ed1b0f18f7e4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:46.097916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:46.098474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:46.098760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:46.099046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:46.107301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:46.108836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:18:46.121754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:46.123302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:46.124417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 615/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:18:46.128950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:46.262227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd680e-3342-4c74-ae81-e40cd7d9a193 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:46.285602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:46.298660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:46.299315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:46.299596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:46.299897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:46.300661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:46.300970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:46.302389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c9f74-04e0-4d57-83eb-9d4654df628e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:46.314580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 603/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:46.321404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:46.433861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3df1d-2bc1-44d6-8cea-5af7af5bf9ac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:46.463894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:46.470410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:46.471186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:46.471450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:46.471926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:46.472820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=2892e755ce4743adbd131ebda92931d4) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:46.473632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:46.475295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9680e7-bcf2-43c7-ac58-85793f3b3a53 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:46.482478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 616/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:18:46 2022] GET /identity/v3/role_assignments?user.id=2892e755ce4743adbd131ebda92931d4&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:18:48.543661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b404-d940-4741-961b-9fffb6ac2bfe None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:48.544229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b404-d940-4741-961b-9fffb6ac2bfe None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:48.544229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b404-d940-4741-961b-9fffb6ac2bfe None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:48.545589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 604/1220] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:18:48.551043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:48.551489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:48.551590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:48.571225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:18:48.572481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:18:48.648234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 617/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:18:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:18:48.673242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9b404-d940-4741-961b-9fffb6ac2bfe None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:48.793082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9b404-d940-4741-961b-9fffb6ac2bfe None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:48.831474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:48.832150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:48.832431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:48.832765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:48.844500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:18:48.849851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:48.851051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:48.851956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 605/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:18:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:48.857482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:48.977141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b15307-81b7-430e-8d92-ab8b0acd619f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.003867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:49.015886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.016505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.016790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.017087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.017767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:49.018083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.019765 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.028311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 618/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:18:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:49.034384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:49.083489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:49.149456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d708b-1d09-4365-90d0-f0ccd31cf35a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.188574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.189306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.189652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.189994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.200777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.202219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:49.217378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:49.218615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:49.219712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:49.219712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:49.219712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:49.219712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:49.219712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 606/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:49.224879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:49.335593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8112b3f-8942-43e2-b526-96f5a26cb68f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.369364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.369980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.370258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.370528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.371207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:49.371509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.372893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.385210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 619/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:49.390850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:49.491960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:49.500833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f03d-2dc9-4800-831c-2b77ee7d5755 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.535700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.536365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.536643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.536916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.544856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.546291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:18:49.557730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:49.558919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:49.560093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 607/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:49.564731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:49.612912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:49.669889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967687-cd18-448a-9136-ed62946824b7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.708395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.709117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.709478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.709758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.710562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:49.710875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.712436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.731634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 620/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:49.738060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:49.854388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd42a-79bc-4bef-b767-409532a95635 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:49.889704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:49.890276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:49.890556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:49.890831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:49.902258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:49.903772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:49.918380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:49.919455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:49.920384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:49.920384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:49.920384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:49.920384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:49.920384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 608/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:49.925453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:50.034285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b58c0c-c86f-4cdb-815c-5f4a35c178e1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:50.036601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:50.070044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:50.070754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:50.071103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:50.071382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:50.072325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:50.072635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:50.074175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:50.086856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 621/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:18:50.093188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:50.104524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:50.205509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca723c2-3d2e-4fcd-b529-5b1d9a2a765d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:50.244646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:50.245287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:50.245563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:50.245877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:50.252758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:50.254383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:18:50.267448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:50.268643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:50.269547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 609/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:18:50.274635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:50.385524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdd1d0-7197-4518-9e92-021d38dadd85 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:50.423105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:50.423725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:50.424037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:50.424317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:50.425066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:50.425377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:50.426863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b883c-783b-4bf2-8270-43344ded006c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:50.440298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 622/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:50.447190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:50.519527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:50.558142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a9fc-49ec-41bd-978a-3e405cc76447 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:50.595242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:50.595940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:50.596223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:50.596495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/2892e755ce4743adbd131ebda92931d4/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:50.612111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=2892e755ce4743adbd131ebda92931d4, role_id=59d1e5b92ab1414e87f8bf0f6f7cbb48)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:50.613641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514b48a-e2f3-4019-90c3-252342351b07 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:50.626693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 610/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:18:50 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/2892e755ce4743adbd131ebda92931d4/roles/59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:18:52.804985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095fc5-5aab-46bd-a7f6-b6b358136f37 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:52.805527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095fc5-5aab-46bd-a7f6-b6b358136f37 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:52.805776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095fc5-5aab-46bd-a7f6-b6b358136f37 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:52.807475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 623/1233] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:18:52.813474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:52.813911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:52.814034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:52.832667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:18:52.833961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:18:52.902978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 611/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 15:18:52.927022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5095fc5-5aab-46bd-a7f6-b6b358136f37 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:52.952838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:53.034729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5095fc5-5aab-46bd-a7f6-b6b358136f37 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.070555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.071121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.071427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.071701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.081794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:18:53.087382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:53.088395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:53.089480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 624/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 15:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:18:53.094832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:53.144789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:53.204882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fa516-2ec2-40bc-b37d-9f1cc9d231af None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.242638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.243396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.243680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.243985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.244703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:53.245010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:53.246388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:53.255460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 612/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:53.260833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:53.349485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:53.366564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e795-bd3b-441b-9911-b7c11e13558f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.402044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.402706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.403067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.403346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.415652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:53.417124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:53.432118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:53.433243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:53.434152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:53.434152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:53.434152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:53.434152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:53.434152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 625/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:53.439336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:53.544221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20bceb-bffd-4b51-8537-d188932d9607 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.553913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:53.581016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.581662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.581970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.582226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.582912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:53.583222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:53.584735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:53.597990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 613/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:53.603794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:53.715081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4196-b0ea-4c96-890b-264442669f38 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.755715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.756514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.756829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.757124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.761152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:53.767395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:53.768984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:18:53.781927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:53.783003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:53.784535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 626/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:53.789144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:53.893996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91d4d-035a-44f1-9aa1-8295edc0e0fc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:53.931536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:53.932215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:53.932500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:53.932826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:53.933754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:53.934113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:53.935610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:53.955085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 614/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:53.960829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:54.075989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b234-e299-4be2-8197-88606774ca14 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:54.120247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:54.120867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:54.121212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:54.121628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:54.187696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:54.195397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:54.196938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:54.304417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 627/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:18:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 15:18:54.310841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:54.339737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:54.429027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492cc6-38f4-41ea-a9cc-5ca1e60c1501 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:54.465525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:54.466081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:54.466357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:54.466627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:54.478513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:54.480386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:54.498720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:54.499986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:54.501067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:54.501067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:54.501067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:54.501067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:54.501067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 615/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:54.506780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:54.617169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:54.623899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520865-4b18-4756-97bd-9f15c090c31b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:54.661054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:54.661675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:54.661957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:54.662231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:54.662917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:54.663234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:54.664640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:54.677098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 628/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:54.682702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:54.766516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:54.792540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3623d-5797-491f-9103-8c6281660f23 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:54.832203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:54.832980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:54.833362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:54.833645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:54.842336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:54.844157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:18:54.857604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:54.858828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:54.859991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 616/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:54.865699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:54.980761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deed8ee-9818-42bb-8f13-eed1bd49ee3a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:55.018485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:55.019060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:55.019351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:55.019673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:55.020467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:55.020772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:55.022341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7575-3b92-45d8-ab67-485a420abd41 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:55.036260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 629/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:18:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:55.043001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:55.157359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1a69-f35d-4fb7-b3a4-f6fd949137f0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:55.195107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:55.195725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:55.196050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:55.196878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:55.197679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] RBAC: Adding query filter params (role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48, scope.project.id=3a947acc76624771ba3560291d433410, user.id=2892e755ce4743adbd131ebda92931d4) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:55.197987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:55.199419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb335f8c-4c98-4bb4-807f-558d145e43ff None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:55.205415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 617/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:18:55 2022] GET /identity/v3/role_assignments?user.id=2892e755ce4743adbd131ebda92931d4&scope.project.id=3a947acc76624771ba3560291d433410&role.id=59d1e5b92ab1414e87f8bf0f6f7cbb48 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:57.515315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3f64-5bae-4373-8341-505159dcdd72 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:57.516046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3f64-5bae-4373-8341-505159dcdd72 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:57.516217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3f64-5bae-4373-8341-505159dcdd72 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:57.517610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 630/1247] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:18:57.523760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:57.524889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:57.524889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:57.531915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:57.546769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:18:57.548099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:18:57.623713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 618/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:18:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:18:57.649811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3f64-5bae-4373-8341-505159dcdd72 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:57.670035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:57.766971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3f64-5bae-4373-8341-505159dcdd72 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:57.809400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849958c1-a670-4839-869f-9323c23106ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:57.810236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849958c1-a670-4839-869f-9323c23106ea None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:57.810561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849958c1-a670-4839-869f-9323c23106ea None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:57.810981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849958c1-a670-4839-869f-9323c23106ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:57.824658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849958c1-a670-4839-869f-9323c23106ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-849958c1-a670-4839-869f-9323c23106ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:18:57.830888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:57.831969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:57.833158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 631/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:18:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:57.839510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:57.955987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265ba0-1432-42e6-ac25-ef571d0a9944 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:57.990533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:57.995988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:57.996625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:57.996914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:57.997195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:57.997972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:57.998347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:57.999660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:58.011872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 619/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:18:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:58.015251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849958c1-a670-4839-869f-9323c23106ea None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:58.101332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:58.127437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849958c1-a670-4839-869f-9323c23106ea None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:58.166443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:58.167217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:58.167498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:58.167771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:58.179105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:58.181284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:58.198940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:58.200065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:58.201199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:58.201199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:58.201199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:58.201199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:58.201199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 632/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:58.206355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:58.322687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61f6a-0f20-490f-91fe-f074e14576ed None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:58.360563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:58.361124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:58.361434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:58.361750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:58.362482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:58.362818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:58.364308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:58.377750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 620/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:18:58.383520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:58.506231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15adf7d-0845-43dc-8a93-10792e5a22af None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:58.535892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:58.545801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:58.546468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:58.546756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:58.547038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:58.556569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:58.558003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:18:58.570632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:58.571881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:18:58.572838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 633/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:58.577903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:58.641179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:58.695083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dda20-166c-4d77-a4fb-3b6554b1b84c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:58.737602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:58.738227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:58.738516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:58.738792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:58.739595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:58.740006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:58.741490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:58.767341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 621/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:18:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:58.768873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:58.891452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e666c6-17cb-4e1d-995a-7f05609667af None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:58.931612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:58.932321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:58.932704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:58.933163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:58.997228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:59.009891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.011365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.121373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 634/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:18:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 15:18:59.127717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:59.237480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cb716-9e46-40c5-8b8f-991761afaf4b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:59.252498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:59.277914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:59.278563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:59.278861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:59.279196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:59.290329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.292045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:18:59.308344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:59.309408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:59.310336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:18:59.310336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:18:59.310336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:18:59.310336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:59.310336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 622/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:59.315103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:59.411043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:59.429111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ea24-b69b-4005-b992-055eb718817f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:59.465837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:59.466500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:59.466785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:59.467064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:59.467752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:59.468254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.469669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.487697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 635/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:18:59.488670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:59.593917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca43b-96ca-43c1-863f-02fc023f5d37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:59.630271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:59.630914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:59.631193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:59.631469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:59.638318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.639780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:18:59.652090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:18:59.653221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:18:59.654308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 623/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:18:59.659335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:59.771635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad5f31-3997-4152-83d3-41ce1429aa43 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:59.812026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:59.812636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:59.812913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:59.813179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:59.813914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:59.814218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.815628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7f7ea-cadf-4db8-a3c9-98501f337b4c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.829071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 636/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:18:59.835899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:18:59.862927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:18:59.943671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b182-4d30-4e2c-9604-c48e08d1e408 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:18:59.983741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d142597-63c6-4666-b794-05660f505602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:18:59.984505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142597-63c6-4666-b794-05660f505602 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:18:59.984785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142597-63c6-4666-b794-05660f505602 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:18:59.985065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142597-63c6-4666-b794-05660f505602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:18:59.985851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d142597-63c6-4666-b794-05660f505602 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=2892e755ce4743adbd131ebda92931d4) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:18:59.986159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d142597-63c6-4666-b794-05660f505602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:18:59.987598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d142597-63c6-4666-b794-05660f505602 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:18:59.993782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 624/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:18:59 2022] GET /identity/v3/role_assignments?user.id=2892e755ce4743adbd131ebda92931d4&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:02.016620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308d7c-0332-4b57-9545-17815e42741e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.017088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308d7c-0332-4b57-9545-17815e42741e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.017179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308d7c-0332-4b57-9545-17815e42741e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.018501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 637/1261] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:02.023521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.023902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.024149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.041825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:02.043013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:02.108163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 625/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:02.130561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4308d7c-0332-4b57-9545-17815e42741e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:02.141947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:02.231671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4308d7c-0332-4b57-9545-17815e42741e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:02.264898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:02.265529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.265812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.266089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.276026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:19:02.280982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:02.282054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:02.283247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 638/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:19:02.288109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:02.306909 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:02.384604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723969a-0a36-4163-9ec6-97a98ce2beeb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:02.418162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:02.418869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.419336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.419695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.420787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:02.421319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:02.425808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:02.437018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 626/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:02.442950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:02.507385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:02.539847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d869fdf-5ee4-4869-b30e-a005df989f5f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:02.572781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:02.573404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.573683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.573954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.583617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:02.585523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:19:02.599414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:02.600477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:02.601448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:19:02.601448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:19:02.601448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:02.601448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:02.601448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 639/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 15:19:02.606724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:02.687735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:02.707111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb16-fae6-49bb-bef5-3d016c137d07 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:02.744709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:02.745402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.745736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.746021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.746796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:02.747144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:02.748812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:02.761782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 627/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:02.768828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:02.874276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041943a5-a029-49b4-aa76-bb1db0be8583 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:02.892886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:02.907053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:02.907680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:02.908000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:02.908275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:02.916311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:02.917729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:19:02.928810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:02.929887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:02.930846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 640/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:02.935886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.037582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f732f-418e-4991-89d3-a3a8136f9751 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.069820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.070421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.071249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.071525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.072334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:03.072641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.074061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.082205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:03.091089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 628/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:19:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:03.096823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.193522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c0de-f026-48d7-8d76-e47fcb2219f3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.226533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.227193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.227479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.227755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.238646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.240123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:19:03.254072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:03.255096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:03.256223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:19:03.256223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:19:03.256223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:03.256223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:03.256223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 641/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:03.261695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.357687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d8756-51d3-4d2d-a263-6d7e52b55328 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.390764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.391371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.391655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.391964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.392653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:03.392957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.394325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.405068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 629/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:03.410815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.442136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:03.509627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146fbf6-b565-4373-8ff2-14d9db97dbbb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.541178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.541800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.542083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.542353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.548799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.550220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:19:03.561862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:03.562895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:03.563875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 642/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:19:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:03.568910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.625576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:03.665439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deec83-752f-4d1d-a854-1ddc645d2a14 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.698141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.698747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.699026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.699300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.700065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:03.700371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.701793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466721c8-8aa5-4371-aac9-2939c79a6920 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.713156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 630/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:19:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:03.719077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:03.813151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:03.822360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad870-24fd-49f9-952a-5d9d23b14c95 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:03.855266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:03.855953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:03.856242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:03.856520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] PATH_INFO: `/v3/projects/3a947acc76624771ba3560291d433410/users/2892e755ce4743adbd131ebda92931d4/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:03.858433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] RBAC: Authorizing `identity:create_grant(project_id=3a947acc76624771ba3560291d433410, user_id=2892e755ce4743adbd131ebda92931d4, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:03.859871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe743a-9002-47b8-a3d9-1a1513f2af8b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:03.872496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 643/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 15:19:03 2022] PUT /identity/v3/projects/3a947acc76624771ba3560291d433410/users/2892e755ce4743adbd131ebda92931d4/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:19:05.837253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74816db4-eb5f-41fa-832a-062f65635c97 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:05.837808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74816db4-eb5f-41fa-832a-062f65635c97 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:05.837808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74816db4-eb5f-41fa-832a-062f65635c97 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:05.839149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 631/1274] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:05.844101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:05.844519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:05.844660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:05.863063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:05.864262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:05.928598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 644/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:05.952062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74816db4-eb5f-41fa-832a-062f65635c97 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.051151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74816db4-eb5f-41fa-832a-062f65635c97 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.068873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:06.085791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.086470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.086756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.087041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.097228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 15:19:06.102633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:06.103706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:06.104969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 632/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 15:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:06.109748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.189533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:06.207668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb090d-9131-4c06-8a8c-18bbd3c42416 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.240688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402590-7474-4d72-8d00-494016581836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.241297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402590-7474-4d72-8d00-494016581836 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.241584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402590-7474-4d72-8d00-494016581836 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.241857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402590-7474-4d72-8d00-494016581836 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.243202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402590-7474-4d72-8d00-494016581836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:06.243499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402590-7474-4d72-8d00-494016581836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:06.244881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402590-7474-4d72-8d00-494016581836 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:06.252685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 645/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:06.258359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.355095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2990b-f8f1-492e-a2a5-f985e03bc8c0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.387601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.388248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.388530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.388804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.398863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:06.400344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:19:06.414954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:06.416027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:06.417239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:19:06.417239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:19:06.417239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:06.417239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:06.417239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 633/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:06.421732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402590-7474-4d72-8d00-494016581836 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.520517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402590-7474-4d72-8d00-494016581836 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.553598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.554200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.554482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.554792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.555472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:06.555776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:06.557151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:06.569329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 646/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:06.575115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.605221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:06.671364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a85e4-0ad8-4562-8ff0-d586671c43f9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.702839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.703459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.703738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.704033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.711571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:06.713038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 15:19:06.724289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:06.725326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:06.727060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 634/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:06.732559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.736787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:06.831517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379c100-6eb6-4c7b-8719-0b920122e5ad None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:06.864267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:06.864828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:06.865079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:06.865352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:06.866128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:06.866436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:06.867878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:06.884878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 647/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:06.890553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:06.978751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:06.988358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244fb96-f578-4220-b3b1-144aa7f92a3a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.020630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.021235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.021513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.021788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.086637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.088191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.171626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:07.182438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 635/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 15:19:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 15:19:07.188432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:07.248338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:07.285922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29730a4-8673-49b9-824c-d5c15aa70079 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.318673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.319302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.319580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.319868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.330185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.331600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 15:19:07.345888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:07.346941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:07.348079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 15:19:07.348079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 15:19:07.348079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 15:19:07.348079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:07.348079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 648/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:07.354283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:07.449329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b41b-b051-489d-a679-fc494ebbd1a2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.481917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.482517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.482797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.483071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.483750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:07.484080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.485439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.496650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 636/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 15:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:07.502062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:07.598684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991d4d6-8177-4478-802c-63cb702643f2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.611631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:07.631010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.631627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.631932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.632206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.639177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.640707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 15:19:07.652481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:07.653514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 27 15:19:07.654646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 649/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 15:19:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:07.661009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:07.715606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:07.768770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88fe4e-6d6a-4291-a471-705a2a008c77 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.800735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.801336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.801610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.801880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.802620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:07.802920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.804400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a24be2-caf1-4858-82c2-2299174bd6e6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.815665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 637/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 15:19:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:07.822472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:07.919004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f4cf8-7add-4fde-b36b-deb23579455c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:07.952543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:07.953146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:07.953427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:07.953701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:07.954475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] RBAC: Adding query filter params (role.id=5ea86f82718f4404bfc74b1781b9f673, scope.project.id=3a947acc76624771ba3560291d433410, user.id=2892e755ce4743adbd131ebda92931d4) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:07.954784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:07.956239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1b474-11b6-4f0e-a7a6-7e408a6e1bb1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:07.962067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 650/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 15:19:07 2022] GET /identity/v3/role_assignments?user.id=2892e755ce4743adbd131ebda92931d4&scope.project.id=3a947acc76624771ba3560291d433410&role.id=5ea86f82718f4404bfc74b1781b9f673 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:09.943762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfa9e-c35d-4a4d-b33c-a078d7324916 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:09.944313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfa9e-c35d-4a4d-b33c-a078d7324916 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:09.944514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfa9e-c35d-4a4d-b33c-a078d7324916 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:09.945683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 638/1288] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:09.950709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:09.951121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:09.951256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:09.969988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:09.971168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:10.000469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:10.035987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 651/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:10.058841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dfa9e-c35d-4a4d-b33c-a078d7324916 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:10.157860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dfa9e-c35d-4a4d-b33c-a078d7324916 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:10.168681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:10.192357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:10.192988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:10.193272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:10.193550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:10.194222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:10.195544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:10.204294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:10.205533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:10.206444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 639/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:19:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:10.212010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:10.308180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733cb1-3b9e-4372-8c44-facb845d10bb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:10.340801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:10.341411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:10.341690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:10.341961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:10.342628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:10.342933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:10.344314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78542b-6242-4343-84f3-d0265c62f2f2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:10.348938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 652/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 15:19:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:10.355230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:10.451165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e1cf-d29b-42aa-bf2a-34cac7639350 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:10.488113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:10.488728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:10.489011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:10.489288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:10.489966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:10.490271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:10.491640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dc15a-373f-487c-8b5e-46a7262173b6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:10.495973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 640/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 15:19:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:12.461962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad102680-bb08-425f-901e-cc438f1d8b69 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:12.462542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad102680-bb08-425f-901e-cc438f1d8b69 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:12.462640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad102680-bb08-425f-901e-cc438f1d8b69 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:12.464521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 653/1293] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:12.475417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df121b7-adc5-4255-ba89-0161dd9c9268 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:12.475956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df121b7-adc5-4255-ba89-0161dd9c9268 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:12.476110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df121b7-adc5-4255-ba89-0161dd9c9268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:12.494283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2df121b7-adc5-4255-ba89-0161dd9c9268 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:12.495471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2df121b7-adc5-4255-ba89-0161dd9c9268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:12.524203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:12.560648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 641/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:12.583061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad102680-bb08-425f-901e-cc438f1d8b69 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:12.613973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:12.680756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad102680-bb08-425f-901e-cc438f1d8b69 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:12.713226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:12.713764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:12.714043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:12.714313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:12.714993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:12.716362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004a0c1-149c-4039-ae58-e8fcf0b9ff9f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:12.731588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 654/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 15:19:12 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:19:14.715686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcdf5-5f6b-4a6d-a4b5-12ea48dfcc3a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:14.716308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcdf5-5f6b-4a6d-a4b5-12ea48dfcc3a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:14.716308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcdf5-5f6b-4a6d-a4b5-12ea48dfcc3a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:14.717620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 642/1296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:14.722733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:14.723246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:14.723370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:14.740943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:14.742631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:14.811387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 655/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:14.833748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fcdf5-5f6b-4a6d-a4b5-12ea48dfcc3a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:14.930772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fcdf5-5f6b-4a6d-a4b5-12ea48dfcc3a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:14.965369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:14.966052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:14.966280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:14.966552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:14.967174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:14.968578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:14.977312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:14.983243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:14.984438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 643/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:19:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:14.985504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:15.002680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:15.081512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bd25-d002-473d-877c-f514b64e5353 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:15.113775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270f43-5425-4372-9259-566043553f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:15.114373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270f43-5425-4372-9259-566043553f2c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:15.114652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270f43-5425-4372-9259-566043553f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:15.114924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270f43-5425-4372-9259-566043553f2c None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:15.115584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270f43-5425-4372-9259-566043553f2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:15.115910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270f43-5425-4372-9259-566043553f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:15.117280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270f43-5425-4372-9259-566043553f2c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:15.122179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 656/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 15:19:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:15.128114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:15.159800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:15.225159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adf541-c2ad-4042-98ce-df0dc1768109 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:15.256441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:15.257034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:15.257850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:15.258120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:15.258860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] RBAC: Adding query filter params (interface=public, service_id=4e38d7d90dbd4dc7bbb92fd3859dc0cc, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:15.259189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:15.260622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f77126-5b24-4ecc-bdfe-ed4c7f0c7a51 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:15.265427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 644/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 15:19:15 2022] GET /identity/v3/endpoints?service_id=4e38d7d90dbd4dc7bbb92fd3859dc0cc&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:19:15.271087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270f43-5425-4372-9259-566043553f2c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:15.349743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:15.368102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270f43-5425-4372-9259-566043553f2c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:15.400831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:15.401439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:15.401720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:15.401992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:15.402634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:15.404004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9a47b-77d6-4124-a4be-14498038dfea None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:15.410599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 657/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 15:19:15 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:19:17.361857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbae21-d528-4ffe-8808-246262b3e0d3 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:17.362389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbae21-d528-4ffe-8808-246262b3e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:17.362389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbae21-d528-4ffe-8808-246262b3e0d3 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:17.363731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 645/1302] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:19:17.368815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:17.369241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:17.369372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:17.388427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:19:17.389620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:19:17.453985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 658/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:19:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 15:19:17.476529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbae21-d528-4ffe-8808-246262b3e0d3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:17.577585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:17.584590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbae21-d528-4ffe-8808-246262b3e0d3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:17.617580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:17.618154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:17.618436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:17.618710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:17.619331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:17.620746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 15:19:17.629487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 15:19:17.630643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 27 15:19:17.631517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 646/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 15:19:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 15:19:17.637193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:17.710714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:17.734412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362b479-0ae1-4034-b4d3-12befc0758c5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:17.772701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:17.773312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:17.773596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:17.773870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:17.774539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:19:17.774867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:17.776274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bbaa9-6130-4088-b0be-097927c8ad50 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:17.781291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 659/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 15:19:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:19:17.787069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:19:17.884017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683aea4-26c1-40d1-bae5-3164dde9df3d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:19:17.915840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:19:17.916449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:19:17.916729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:19:17.917015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:19:17.917696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:19:17.919054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f307c8-8f3b-420a-959c-4bde031266de None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:19:17.938281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 647/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:19:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:19:27.778314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=30 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:19:27.933999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:20.078146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1abca7-ff18-48ca-b35c-ae74b9fb772f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:20.078689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1abca7-ff18-48ca-b35c-ae74b9fb772f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:20.078689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1abca7-ff18-48ca-b35c-ae74b9fb772f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:20.083782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 660/1307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:20.085197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b31-12ac-4bbf-a220-7c87f0ccca77 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:20.085664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b31-12ac-4bbf-a220-7c87f0ccca77 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:20.085749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b31-12ac-4bbf-a220-7c87f0ccca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:20.200783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-475f7b31-12ac-4bbf-a220-7c87f0ccca77 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:20.202495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-475f7b31-12ac-4bbf-a220-7c87f0ccca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:20.351866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 648/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:20.393473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:20.394117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:20.394630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:20.396643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 661/1309] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 15:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:20.403290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9e497-8128-4720-9951-7c421962385c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:20.403607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9e497-8128-4720-9951-7c421962385c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:20.403898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9e497-8128-4720-9951-7c421962385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:20.490109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cce9e497-8128-4720-9951-7c421962385c None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:20.491337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cce9e497-8128-4720-9951-7c421962385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:20.519096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:20.612963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 649/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:20.622487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:20.624628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:20.819918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06c4f5-b322-434b-816f-498ad721420d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:20.905048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:20.905670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:20.905949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:20.906223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:20.907870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:20.970273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:21.040317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:21.041956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749720a-2d1a-4922-9b76-f48fdcfe1845 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:21.064337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 662/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:23.470095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:23.470641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:23.470641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:23.472192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 650/1312] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:23.477147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab081-b809-4b79-8407-31a930617b43 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:23.477573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab081-b809-4b79-8407-31a930617b43 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:23.477742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab081-b809-4b79-8407-31a930617b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:23.572988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-19cab081-b809-4b79-8407-31a930617b43 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:23.574299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19cab081-b809-4b79-8407-31a930617b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:23.687681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 663/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:23.726540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:23.728159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:23.853001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90111-07e1-43ea-b28f-285f37078506 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:23.930467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:23.933628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:23.934664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:23.935594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:23.935594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:23.937624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:24.066821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:24.068711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a3651-2248-422a-82d1-35233ccccbea service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:24.088130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 651/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:26.438412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:26.438978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:26.438978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:26.441572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 664/1315] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:26.446135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8deb7c-9695-4bb2-b9a1-3030f66593bb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:26.446573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8deb7c-9695-4bb2-b9a1-3030f66593bb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:26.446713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8deb7c-9695-4bb2-b9a1-3030f66593bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:26.511005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9c8deb7c-9695-4bb2-b9a1-3030f66593bb None None] MFA Rules not processed for user `fc52d58399b846da8fd7d4217e87ec87`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:26.512227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9c8deb7c-9695-4bb2-b9a1-3030f66593bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:26.561753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:26.607644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 652/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:26.647759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:26.649317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:26.659008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:26.750865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae788cd-96b8-41d0-8e87-696583d1d639 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:26.823895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:26.824508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:26.824785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:26.825054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:26.826668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:26.887754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:26.933794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:26.935208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dda025-3b81-4666-a691-5b0e3f3f1e81 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:26.953847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 665/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:20:29.512950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:29.513490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:29.513490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:29.514853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 653/1318] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:29.520744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232842-884b-47d7-82e1-16bfa37fae3b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:29.521146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232842-884b-47d7-82e1-16bfa37fae3b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:29.521345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232842-884b-47d7-82e1-16bfa37fae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:29.587638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e8232842-884b-47d7-82e1-16bfa37fae3b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:29.588899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e8232842-884b-47d7-82e1-16bfa37fae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:29.683295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 666/1319] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:29.721368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:29.722892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:29.827008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9afd-57fe-46db-a01d-c6ddfb6204ac None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:29.897730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:29.898336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:29.898612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:29.898884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:29.900521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:29.907759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:30.010214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:30.011655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c62179-8b26-4dc4-bdea-d339b8fd50c0 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:30.029144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 654/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:32.677409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70e54-e24c-4498-ac53-abc2e6a62c85 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:32.677947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70e54-e24c-4498-ac53-abc2e6a62c85 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:32.677947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70e54-e24c-4498-ac53-abc2e6a62c85 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:32.679329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 667/1321] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:32.684442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7899a-97ac-42f6-8057-0e2bde15e8ab None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:32.684901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7899a-97ac-42f6-8057-0e2bde15e8ab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:32.685000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7899a-97ac-42f6-8057-0e2bde15e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:32.753598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7ee7899a-97ac-42f6-8057-0e2bde15e8ab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:32.754782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7ee7899a-97ac-42f6-8057-0e2bde15e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:32.814865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:32.859588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 655/1322] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:32.913847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:32.914136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:32.914412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:32.915793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 668/1323] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 15:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:32.922294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f23-ef6d-441c-9d11-cdbf44f59061 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:32.922574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f23-ef6d-441c-9d11-cdbf44f59061 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:32.922845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f23-ef6d-441c-9d11-cdbf44f59061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:33.004449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-165c5f23-ef6d-441c-9d11-cdbf44f59061 None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:33.005906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-165c5f23-ef6d-441c-9d11-cdbf44f59061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:33.083763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 656/1324] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:33.092606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:33.094228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:33.115230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:33.197697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b877ca-f09c-4092-85e5-43763d28d95f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:33.269870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:33.270380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:33.270664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:33.270940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:33.272615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:33.354390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:33.393534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:33.394970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b641e9-d877-475a-9e9e-369ed1de256d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:33.413449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 669/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:20:35.705275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:35.705783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:35.705888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:35.707164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 657/1326] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:35.712257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d1c00-01a8-4aa5-96ab-85c8505ded37 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:35.712705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d1c00-01a8-4aa5-96ab-85c8505ded37 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:35.712830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d1c00-01a8-4aa5-96ab-85c8505ded37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:35.786751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6c7d1c00-01a8-4aa5-96ab-85c8505ded37 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:35.788241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6c7d1c00-01a8-4aa5-96ab-85c8505ded37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:35.888369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 670/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:35.925305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:35.926902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:35.995718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:36.039107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99b7a-c14f-48a0-a335-719d8cbd29b2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:36.110766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:36.111390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:36.111673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:36.111992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:36.113624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:36.222279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:36.224818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:36.226638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfb86d-caf4-4efd-b7d3-b48bff7e7ab5 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:36.244628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 658/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:38.280189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:38.280739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:38.280739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:38.282085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 671/1329] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:38.287245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e899-77fa-4d21-b404-bd7e7774fae9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:38.287702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e899-77fa-4d21-b404-bd7e7774fae9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:38.287959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e899-77fa-4d21-b404-bd7e7774fae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:38.354046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ce18e899-77fa-4d21-b404-bd7e7774fae9 None None] MFA Rules not processed for user `fc52d58399b846da8fd7d4217e87ec87`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:38.355261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ce18e899-77fa-4d21-b404-bd7e7774fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:38.451924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 659/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:38.489317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:38.490874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:38.537448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:38.623043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480b34e-7bed-47de-9366-666df2560cbd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:38.699072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:38.699799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:38.700117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:38.700390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:38.702012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:38.778135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:38.810830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:38.812577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d9e6-bbe2-4784-bac3-6933eed686ab service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:38.830933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 672/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:41.082547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:41.083086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:41.083086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:41.084466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 660/1332] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:41.089526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906416-3ae5-43e7-83cf-d9a834c9ef04 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:41.089912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906416-3ae5-43e7-83cf-d9a834c9ef04 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:41.090077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906416-3ae5-43e7-83cf-d9a834c9ef04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:41.154983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-49906416-3ae5-43e7-83cf-d9a834c9ef04 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:41.156206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-49906416-3ae5-43e7-83cf-d9a834c9ef04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:41.251802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 673/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:41.287943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:41.289474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:41.358467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:41.393602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dae65-5e3e-48e6-a841-4c927dbb9582 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:41.465529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:41.466142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:41.466425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:41.466701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:41.468356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:41.573027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:41.574375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e77d-db34-451f-b794-dd8796962b2e service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:41.584649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:41.592619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 661/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:43.613310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:43.613848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:43.613848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:43.615215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 674/1335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:43.621547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ba663-c2d7-41aa-a49f-3351027e2f18 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:43.621962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ba663-c2d7-41aa-a49f-3351027e2f18 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:43.622156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ba663-c2d7-41aa-a49f-3351027e2f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:43.688618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-328ba663-c2d7-41aa-a49f-3351027e2f18 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:43.689815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-328ba663-c2d7-41aa-a49f-3351027e2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:43.787161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 662/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:43.827689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:43.829259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:43.870604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:43.932817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128abc4-a3d9-478f-a7bb-cd97fcf1df58 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:44.006572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:44.007184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:44.007461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:44.007731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:44.009372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:44.090647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:44.116315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:44.117757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90740ae-52ee-4d29-8861-be8737de29be service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:44.135037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 675/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:46.494874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:46.495474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:46.495474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:46.496829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 663/1338] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:46.502689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88514-74fc-4792-a9ee-81feee0afab9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:46.503128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88514-74fc-4792-a9ee-81feee0afab9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:46.503235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88514-74fc-4792-a9ee-81feee0afab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:46.569631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e4b88514-74fc-4792-a9ee-81feee0afab9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:46.570838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e4b88514-74fc-4792-a9ee-81feee0afab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:46.666746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 676/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:46.703150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:46.704744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:46.787669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:46.815611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3783b8-106e-4e21-a02f-29a629f9520d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:46.885794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:46.886406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:46.886686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:46.886961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:46.888613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:47.002227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:47.003688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363c316-1468-4e8b-98e6-0be4d47cd828 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:47.021827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 664/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:50.883631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:50.884164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:50.884274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:50.885723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 677/1341] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:50.891414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e736e4-83a9-42a8-aa17-61d2c2a30d46 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:50.891896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e736e4-83a9-42a8-aa17-61d2c2a30d46 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:50.891981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e736e4-83a9-42a8-aa17-61d2c2a30d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:50.897432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:50.965755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-93e736e4-83a9-42a8-aa17-61d2c2a30d46 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:50.966963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93e736e4-83a9-42a8-aa17-61d2c2a30d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:51.064692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 665/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:51.102910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:51.104598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:51.154098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:51.209882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943c82-5e79-48ce-ad07-a7759e2e2dbd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:51.284273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:51.284994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:51.285343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:51.285678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:51.287540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:51.381416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:51.397996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:51.399420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cefa7b-09dd-4f56-85fc-e8ca25238475 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:51.417734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 678/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:20:54.379612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:54.380199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:54.380199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:54.381542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 666/1344] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:54.386780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d074f0-8cdd-4e9c-9ddf-8316df5b0edb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:54.387208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d074f0-8cdd-4e9c-9ddf-8316df5b0edb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:54.387344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d074f0-8cdd-4e9c-9ddf-8316df5b0edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:54.453308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c9d074f0-8cdd-4e9c-9ddf-8316df5b0edb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:54.454435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c9d074f0-8cdd-4e9c-9ddf-8316df5b0edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:54.549844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 679/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:54.588006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:54.589548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:54.677315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:54.695604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d52654-886f-40c3-b0b1-87ead067a47f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:54.772484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:54.773109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:54.773387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:54.773663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:54.775273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:54.889965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:54.891411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad5aa-e226-4f96-9add-fe0bac6fecd7 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:54.910150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 667/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:20:59.110006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:59.110598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:59.110598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:59.111955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 680/1347] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:20:59.117207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f661c-3d11-4a19-aa5f-46fc1bb40d67 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:59.117685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f661c-3d11-4a19-aa5f-46fc1bb40d67 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:59.117766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f661c-3d11-4a19-aa5f-46fc1bb40d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:59.134072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:59.185991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-704f661c-3d11-4a19-aa5f-46fc1bb40d67 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:20:59.187969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-704f661c-3d11-4a19-aa5f-46fc1bb40d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:59.288083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 668/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:20:59.326023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:20:59.327734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:59.384996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:59.430919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284cdd2-9178-4a16-9698-684b04d219d9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:20:59.509287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:20:59.509816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:20:59.510099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:20:59.510374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:20:59.512236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:20:59.617352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:20:59.621530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:20:59.622947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9db8f-793b-4e71-820c-bf92505f8172 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:20:59.640951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 681/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:04.465254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9333e6-f8b6-48f9-a89a-9597b2ad982a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:04.465785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9333e6-f8b6-48f9-a89a-9597b2ad982a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:04.465785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9333e6-f8b6-48f9-a89a-9597b2ad982a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:04.467572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 669/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:04.473859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d5ee-339c-45ca-99eb-e9aef57263f4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:04.474646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d5ee-339c-45ca-99eb-e9aef57263f4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:04.475150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d5ee-339c-45ca-99eb-e9aef57263f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:04.628744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ffc9d5ee-339c-45ca-99eb-e9aef57263f4 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:04.630307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ffc9d5ee-339c-45ca-99eb-e9aef57263f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:04.755110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 682/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:08.818616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887542-051d-45ab-80db-332ef9ba7403 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:08.819529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887542-051d-45ab-80db-332ef9ba7403 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:08.820067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887542-051d-45ab-80db-332ef9ba7403 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:08.821883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 670/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:08.828408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e7d1d-fb45-47ff-b2c5-7e74a4e4cd7a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:08.828892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e7d1d-fb45-47ff-b2c5-7e74a4e4cd7a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:08.829001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e7d1d-fb45-47ff-b2c5-7e74a4e4cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:08.883635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:08.908966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d85e7d1d-fb45-47ff-b2c5-7e74a4e4cd7a None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:08.910168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d85e7d1d-fb45-47ff-b2c5-7e74a4e4cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:09.025605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 683/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:09.286187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=59 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:10.465087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ff24-f713-499e-ae81-e8b68412a954 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:10.465714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ff24-f713-499e-ae81-e8b68412a954 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:10.465714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ff24-f713-499e-ae81-e8b68412a954 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:10.467129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 671/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:10.472132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f998d0-366b-493c-94aa-dfe607378be0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:10.472544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f998d0-366b-493c-94aa-dfe607378be0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:10.472793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f998d0-366b-493c-94aa-dfe607378be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:10.549626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-73f998d0-366b-493c-94aa-dfe607378be0 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:10.550813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-73f998d0-366b-493c-94aa-dfe607378be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:10.593652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:10.657426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 684/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:21:13.965240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce81aa-ca99-4784-97df-05333ce9425f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:13.965240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce81aa-ca99-4784-97df-05333ce9425f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:13.965240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce81aa-ca99-4784-97df-05333ce9425f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:13.966594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 672/1356] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:13.972431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4101d-8a3f-4287-9b6a-ec7cba2891bb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:13.973446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4101d-8a3f-4287-9b6a-ec7cba2891bb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:13.973887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4101d-8a3f-4287-9b6a-ec7cba2891bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:14.063333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-65f4101d-8a3f-4287-9b6a-ec7cba2891bb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:14.064749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-65f4101d-8a3f-4287-9b6a-ec7cba2891bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:14.175940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 685/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:14.228039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:14.228731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:14.229240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:14.230862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 673/1358] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 15:21:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:14.239858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c01bde-a680-41ea-9880-57283eb4e7ec None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:14.240157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c01bde-a680-41ea-9880-57283eb4e7ec None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:14.240438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c01bde-a680-41ea-9880-57283eb4e7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:14.260434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:14.329051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-34c01bde-a680-41ea-9880-57283eb4e7ec None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:14.330309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-34c01bde-a680-41ea-9880-57283eb4e7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:14.454965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 686/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:14.462920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:14.464652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:14.621953 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126fe04-7e90-4fd5-b9ba-74b64850ad2d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:14.698801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:14.741596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:14.742141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:14.742417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:14.742688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:14.744489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:14.839580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:14.841632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bcb16-f0a9-4812-97b6-216eea00af77 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:14.862400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 674/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:21:17.083311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:17.083802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:17.084016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:17.085848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 687/1361] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:17.092535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bcd52-b049-4f13-bd6d-f97d183e8924 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:17.093029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bcd52-b049-4f13-bd6d-f97d183e8924 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:17.093292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bcd52-b049-4f13-bd6d-f97d183e8924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:17.170261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-928bcd52-b049-4f13-bd6d-f97d183e8924 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:17.171370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-928bcd52-b049-4f13-bd6d-f97d183e8924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:17.211767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:17.275263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 675/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:17.323071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:17.324942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:17.401028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:17.508033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33404552-2069-4d2d-904b-9300e282bb3a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:17.663098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:17.663955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:17.664486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:17.664929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:17.666705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:17.695077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:17.774770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:17.776405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592619ba-337a-4e14-b4a7-d9b40dda44c0 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:17.796340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 688/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:20.183302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60290e9-1558-4a7b-a820-d4b82622401b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:20.184050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60290e9-1558-4a7b-a820-d4b82622401b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:20.184050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60290e9-1558-4a7b-a820-d4b82622401b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:20.185256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 676/1364] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:20.190824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62af82-4423-4cb8-9bd7-fcc4c30fc7fe None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:20.191258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62af82-4423-4cb8-9bd7-fcc4c30fc7fe None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:20.191494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62af82-4423-4cb8-9bd7-fcc4c30fc7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:20.262256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8a62af82-4423-4cb8-9bd7-fcc4c30fc7fe None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:20.263989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8a62af82-4423-4cb8-9bd7-fcc4c30fc7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:20.369097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:20.375923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 689/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:20.421061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:20.421591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:20.421979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:20.423581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 677/1366] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 15:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:20.429125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad50bd-f4f7-4710-83b7-326c7286ee58 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:20.429432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad50bd-f4f7-4710-83b7-326c7286ee58 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:20.429830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad50bd-f4f7-4710-83b7-326c7286ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:20.512546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e1ad50bd-f4f7-4710-83b7-326c7286ee58 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:20.514228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1ad50bd-f4f7-4710-83b7-326c7286ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:20.638915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 690/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:20.646845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:20.648492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:20.811276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55292f3d-b97a-4a10-aa35-39af101a3cb7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:20.832441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:20.932955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:20.933578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:20.933858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:20.934133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:20.936400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:21.030502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:21.032149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68205-7a80-4b12-8351-63ef5db2675f service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:21.052172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 678/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:21.972742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d8d3-1e06-4c28-aaf6-7a06edde740f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:21.973145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d8d3-1e06-4c28-aaf6-7a06edde740f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:21.973355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d8d3-1e06-4c28-aaf6-7a06edde740f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:21.974933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 691/1369] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:21.980165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd17e-44a2-487e-a764-4786cede3b47 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:21.980613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd17e-44a2-487e-a764-4786cede3b47 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:21.980695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd17e-44a2-487e-a764-4786cede3b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:22.020136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:22.055971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b81dd17e-44a2-487e-a764-4786cede3b47 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:22.057181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b81dd17e-44a2-487e-a764-4786cede3b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:22.158489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 679/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:22.525523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:22.525907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:22.526299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:22.527954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 692/1371] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 15:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:21:22.532825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dae9b-1633-4387-b949-233b80fe48e0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:22.533225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dae9b-1633-4387-b949-233b80fe48e0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:22.533618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dae9b-1633-4387-b949-233b80fe48e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:22.615286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eb9dae9b-1633-4387-b949-233b80fe48e0 None None] MFA Rules not processed for user `711efe02d15146dfb6a1a737abcee3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:22.616704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eb9dae9b-1633-4387-b949-233b80fe48e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:22.665806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:22.698967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 680/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:22.707227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:22.708821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:22.763499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:22.824250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f84e2d-b987-4de8-9e09-3f3d5f58a0eb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:22.904343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:22.904958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:22.905236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:22.905507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:22.907127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:23.020489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:23.035619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:23.037390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264abd4-f5d0-4bf1-87fe-f4a2bf8e1ccf service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:23.056040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 693/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:21:23.179438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:23.179983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:23.179983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:23.184156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 681/1374] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:23.186538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc644-57f9-4b44-8a7c-d5ef3797f485 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:23.186830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc644-57f9-4b44-8a7c-d5ef3797f485 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:23.187133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc644-57f9-4b44-8a7c-d5ef3797f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:23.277980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-363bc644-57f9-4b44-8a7c-d5ef3797f485 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:23.279892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-363bc644-57f9-4b44-8a7c-d5ef3797f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:23.423726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 694/1375] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:23.468448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:23.470002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:23.616109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e1ae4-ea8e-47e6-9ddf-d59daf865927 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:23.667199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:23.732905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:23.733597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:23.733874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:23.734145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:23.735771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:23.829939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:23.831535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8fa40-8f50-49d0-9004-2ad9f223f5fb service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:23.849264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 682/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:25.876859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86606038-98db-494b-8975-b61621f253a2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:25.877320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86606038-98db-494b-8975-b61621f253a2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:25.877431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86606038-98db-494b-8975-b61621f253a2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:25.878933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 695/1377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:25.887335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e85f6-e335-4cb9-b072-5e3b64624a29 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:25.887335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e85f6-e335-4cb9-b072-5e3b64624a29 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:25.887335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e85f6-e335-4cb9-b072-5e3b64624a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:25.953676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e16e85f6-e335-4cb9-b072-5e3b64624a29 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:25.954900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e16e85f6-e335-4cb9-b072-5e3b64624a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:25.962107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:26.091774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 683/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:26.138405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86606038-98db-494b-8975-b61621f253a2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:26.139917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86606038-98db-494b-8975-b61621f253a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:26.203852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:26.289665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86606038-98db-494b-8975-b61621f253a2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:26.405211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:26.406068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:26.406512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:26.406941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:26.408900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:26.428914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:26.527926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:26.529734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2e132-6a4e-4edd-a933-fffaf853697f service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:26.555997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 696/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:28.618281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:28.618855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:28.618855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:28.620344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 684/1380] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:28.625473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d66d1-8ed5-4ca0-9c1a-a9578354efc2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:28.625877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d66d1-8ed5-4ca0-9c1a-a9578354efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:28.626104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d66d1-8ed5-4ca0-9c1a-a9578354efc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:28.696623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1e5d66d1-8ed5-4ca0-9c1a-a9578354efc2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:28.697819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1e5d66d1-8ed5-4ca0-9c1a-a9578354efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:28.796273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:28.803579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 697/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:28.847772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:28.849469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:29.005506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:29.017814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82362e04-4a0a-4e1b-bc6e-2a5514244fa8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:29.136832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:29.137993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:29.138342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:29.138707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:29.140418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:29.241613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:29.243423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef366-1719-44ed-9385-eaa99998b6f5 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:29.255501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:29.268106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 685/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:31.334061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:31.334609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:31.334609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:31.336028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 698/1383] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:31.341496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fe42-741d-4312-af2a-1709d0914329 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:31.341978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fe42-741d-4312-af2a-1709d0914329 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:31.342076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fe42-741d-4312-af2a-1709d0914329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:31.413951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8da2fe42-741d-4312-af2a-1709d0914329 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:31.415230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8da2fe42-741d-4312-af2a-1709d0914329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:31.522143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 686/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:31.565106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:31.566651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:31.721166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778553c6-7a53-4289-a4ab-4b535f0ccbb3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:31.800244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:31.851056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:31.851698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:31.852020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:31.852448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:31.854019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:31.964745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:31.966345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4b382-98c1-406c-81f3-e1664c01970e service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:31.990563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 699/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:34.098724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:34.099249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:34.099345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:34.100742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 687/1386] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:34.107744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f50e-b56d-4059-bd65-e60ef4d7061b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:34.108192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f50e-b56d-4059-bd65-e60ef4d7061b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:34.108338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007f50e-b56d-4059-bd65-e60ef4d7061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:34.185490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f007f50e-b56d-4059-bd65-e60ef4d7061b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:34.187516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f007f50e-b56d-4059-bd65-e60ef4d7061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:34.234204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:34.312915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 700/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:21:34.380742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:34.382273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:34.463634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:34.555780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e403e-9b7a-470d-ae65-1b8d1a4f2d7c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:34.692150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:34.692682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:34.692965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:34.693242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:34.694908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:34.739478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:34.804466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:34.806157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6ff6d-e5c7-450c-86e7-403141a14cad service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:34.826588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 688/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:36.967315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:36.967748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:36.967874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:36.969239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 701/1389] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:36.974296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf103c0-4b19-4c2f-84d0-b16361c4246c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:36.974732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf103c0-4b19-4c2f-84d0-b16361c4246c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:36.974838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf103c0-4b19-4c2f-84d0-b16361c4246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:37.041111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ddf103c0-4b19-4c2f-84d0-b16361c4246c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:37.042328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ddf103c0-4b19-4c2f-84d0-b16361c4246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:37.139548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 689/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:21:37.182554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:37.184139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:37.339371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ea25-729c-4f70-a1a3-c3dbb8070d4f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:37.356250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:37.451654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:37.452296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:37.452574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:37.452844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:37.454443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:37.542998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:37.544642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f0491-a442-487e-afc6-b2d353e26487 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:37.563703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 702/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:38.107205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6662a6-5560-4ee2-8394-a0e8b360373d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:38.107768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6662a6-5560-4ee2-8394-a0e8b360373d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:38.107768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6662a6-5560-4ee2-8394-a0e8b360373d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:38.109299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 690/1392] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:38.114234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc291-11dd-4b5b-8f1a-7b78d90d0cf2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:38.114627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc291-11dd-4b5b-8f1a-7b78d90d0cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:38.114816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc291-11dd-4b5b-8f1a-7b78d90d0cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:38.141145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:38.182645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4bcbc291-11dd-4b5b-8f1a-7b78d90d0cf2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:38.184434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4bcbc291-11dd-4b5b-8f1a-7b78d90d0cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:38.282452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 703/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:38.337779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:38.338070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:38.338343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:38.339686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 691/1394] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 15:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:38.345242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5b755-607a-4cb6-931d-130cacfa2747 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:38.345533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5b755-607a-4cb6-931d-130cacfa2747 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:38.345808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5b755-607a-4cb6-931d-130cacfa2747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:38.421431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-05d5b755-607a-4cb6-931d-130cacfa2747 None None] MFA Rules not processed for user `77f96ae3547b4be6bd6bb31c14903926`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:38.422652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-05d5b755-607a-4cb6-931d-130cacfa2747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:38.462271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:38.498342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 704/1395] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:38.505748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:38.507262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:38.520680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:38.612233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de216d4e-ea63-4b09-9d10-46024a4799e7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:38.687238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:38.687906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:38.688190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:38.688466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:38.690174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:38.759978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:38.801692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:38.803171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f9ffc-eeb5-4391-8b61-e04e58a07232 service glance] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:38.821642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 692/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:39.570933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0edce-81f8-4ef7-a498-810dc057fee5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:39.571456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0edce-81f8-4ef7-a498-810dc057fee5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:39.571559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0edce-81f8-4ef7-a498-810dc057fee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:39.597956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556442-6727-4db5-9d32-652ea32888bf None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:39.598497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556442-6727-4db5-9d32-652ea32888bf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:39.598497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556442-6727-4db5-9d32-652ea32888bf None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:39.599932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 693/1397] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:21:39.604944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1380bff-7574-41e6-8fd2-114759e006f3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:39.605233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1380bff-7574-41e6-8fd2-114759e006f3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:39.605548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1380bff-7574-41e6-8fd2-114759e006f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:39.640262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e0c0edce-81f8-4ef7-a498-810dc057fee5 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:39.641465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e0c0edce-81f8-4ef7-a498-810dc057fee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:39.671739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c1380bff-7574-41e6-8fd2-114759e006f3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:39.672970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c1380bff-7574-41e6-8fd2-114759e006f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:39.727455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 705/1398] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:39.773540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 694/1399] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:21:39.776333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:39.776690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:39.777089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:39.778569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 706/1400] 10.222.0.36 () {54 vars in 943 bytes} [Wed Jul 27 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:21:39.785391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:39.785684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:39.785960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:39.812685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:39.818230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:39.819759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:39.863813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] MFA Rules not processed for user `eeec63e358424f57ae02d5693b359805`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:39.865039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:39.882290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:39.971329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9145f4-3c27-4654-90e9-9b08e40c2745 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:39.977515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 695/1401] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Jul 27 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:39.987030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:39.996851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971bbca-abc3-4b3d-abd3-34eed799735d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:40.069123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:40.090861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:40.091479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:40.091754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:40.092053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:40.093682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:40.104993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:40.106204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:40.106488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:40.106766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:40.108446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:40.112758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:40.185401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:40.186905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7f87-3505-4cbf-b193-9831a946b984 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:40.191005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:40.192636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084530d-529a-4c57-b9fa-2f898f20537a service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:40.201293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 696/1402] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 15:21:40.206126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 707/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:21:42.278973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:42.279528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:42.279528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:42.280852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 697/1404] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:42.286094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b96a-3b54-4ffa-9117-9fb233b46ac7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:42.286523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b96a-3b54-4ffa-9117-9fb233b46ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:42.286648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b96a-3b54-4ffa-9117-9fb233b46ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:42.361550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f6a7b96a-3b54-4ffa-9117-9fb233b46ac7 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:42.362846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f6a7b96a-3b54-4ffa-9117-9fb233b46ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:42.462830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:42.471884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 708/1405] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:42.514670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:42.516239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:42.632188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:42.677737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86e193-b6ef-4bff-b42c-4c3d61c60dda None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:42.815967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:42.816585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:42.816863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:42.817134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:42.818754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:42.898061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:42.921018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:42.922662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7cdb8-d2d2-44a3-97b3-24e6ea337aed service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:42.943683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 698/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:45.049888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:45.050482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:45.050482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:45.051794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 709/1407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:21:45.057234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b68d4-e4b5-485d-a244-bf5b78319b56 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:45.057744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b68d4-e4b5-485d-a244-bf5b78319b56 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:45.057851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b68d4-e4b5-485d-a244-bf5b78319b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:45.134030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-682b68d4-e4b5-485d-a244-bf5b78319b56 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:21:45.135287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-682b68d4-e4b5-485d-a244-bf5b78319b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:45.251252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 699/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:21:45.302369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:21:45.303924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:45.499025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1bbd-14fb-46b4-82a3-bf7fb7f0541f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:21:45.598745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:45.665957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:21:45.665957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:21:45.665957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:21:45.666404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:21:45.668716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:21:45.776740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:21:45.779047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f39cfd-e9e8-443b-b38f-98da228173f8 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:21:45.801965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 710/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:21:55.248569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=79 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:21:55.799395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=64 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:02.201592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e2c7-9e27-4b2c-bafb-948a01ba8c0c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:02.202208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e2c7-9e27-4b2c-bafb-948a01ba8c0c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:02.202208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e2c7-9e27-4b2c-bafb-948a01ba8c0c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:02.203582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 700/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 15:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:22:02.208905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beedae-eded-4d38-a5ec-8cb76367bd9f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:02.209354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beedae-eded-4d38-a5ec-8cb76367bd9f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:02.209459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beedae-eded-4d38-a5ec-8cb76367bd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:02.285251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-59beedae-eded-4d38-a5ec-8cb76367bd9f None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:02.286393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-59beedae-eded-4d38-a5ec-8cb76367bd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:02.393322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 711/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:02.407454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee261e-6105-49c6-a0e7-89f5012f9bcd None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:02.407738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee261e-6105-49c6-a0e7-89f5012f9bcd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:02.408044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee261e-6105-49c6-a0e7-89f5012f9bcd None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:02.409386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 701/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:02.414037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12527-0848-4460-a410-1d9890c7bfa4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:02.414323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12527-0848-4460-a410-1d9890c7bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:02.414597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12527-0848-4460-a410-1d9890c7bfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:02.486233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:02.489443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-50e12527-0848-4460-a410-1d9890c7bfa4 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:02.491147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-50e12527-0848-4460-a410-1d9890c7bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:02.606590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 712/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:05.063524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:05.064090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:05.064090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:05.065372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:05.070708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959a571-e5c4-4b6a-b4a0-fa3163f96da2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:05.071122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959a571-e5c4-4b6a-b4a0-fa3163f96da2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:05.071294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959a571-e5c4-4b6a-b4a0-fa3163f96da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:05.136892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3959a571-e5c4-4b6a-b4a0-fa3163f96da2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:05.138092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3959a571-e5c4-4b6a-b4a0-fa3163f96da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:05.192313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:05.234622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:05.279364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:05.281021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:05.428059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612782b-cd45-4c13-9f15-76f91ba52a32 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:05.498728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:05.538255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:05.538867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:05.539142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:05.539412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:05.541052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:05.627055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:05.628671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aa910-df21-44c0-be21-143b953783e9 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:05.646808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 703/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:06.545272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:06.545840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:06.545840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:06.547180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:06.552127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0ae7a-1f4f-4d7f-8f1e-a6b3f30a7548 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:06.552535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0ae7a-1f4f-4d7f-8f1e-a6b3f30a7548 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:06.552650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0ae7a-1f4f-4d7f-8f1e-a6b3f30a7548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:06.627370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c0a0ae7a-1f4f-4d7f-8f1e-a6b3f30a7548 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:06.628614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c0a0ae7a-1f4f-4d7f-8f1e-a6b3f30a7548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:06.658365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:06.727923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:06.878569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:06.880125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:07.023092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c7f3f-fd19-4567-a9df-4278e64d75e4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:07.064083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:07.136183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:07.136728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:07.137010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:07.137785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:07.139432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:07.228262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:07.229829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9ddd2-e716-43fe-a776-4b98bb40e934 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:07.247261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 715/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:08.122994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:08.123542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:08.123542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:08.124901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:08.129740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fda8d-c272-454e-b28e-6c01e58d486b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:08.130169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fda8d-c272-454e-b28e-6c01e58d486b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:08.130303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fda8d-c272-454e-b28e-6c01e58d486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:08.183311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:08.205446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8e8fda8d-c272-454e-b28e-6c01e58d486b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:08.206656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8e8fda8d-c272-454e-b28e-6c01e58d486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:08.303065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:08.455274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:08.457059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:08.599188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18565ba4-1de6-4101-84ad-18fca6e03e8e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:08.611465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:08.710094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:08.710697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:08.710973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:08.711242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:08.712877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:08.799003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:08.800633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2183a-fd91-4c54-a52f-8e998fc97282 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:08.818793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 706/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:09.670541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:09.671091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:09.671091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:09.672481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:09.677452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a438704-abe5-4e36-b9ca-654454dfbc09 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:09.677874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a438704-abe5-4e36-b9ca-654454dfbc09 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:09.678007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a438704-abe5-4e36-b9ca-654454dfbc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:09.697759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:09.754835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9a438704-abe5-4e36-b9ca-654454dfbc09 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:09.756059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9a438704-abe5-4e36-b9ca-654454dfbc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:09.851261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:10.000909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:10.002440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:10.112497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:10.148093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645910-ae73-4919-8d8d-cf55f0a3e88b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:10.258789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:10.259330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:10.259612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:10.259921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:10.261543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:10.338646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:10.352314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:10.353896 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad51751-4dda-4244-a18d-ffc1c17a539d service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:10.372600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 718/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:11.220360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:11.220912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:11.220912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:11.222167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:11.227234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c9c17-394a-4866-8579-f7dd257bf440 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:11.227655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c9c17-394a-4866-8579-f7dd257bf440 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:11.227823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c9c17-394a-4866-8579-f7dd257bf440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:11.304492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-256c9c17-394a-4866-8579-f7dd257bf440 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:11.305697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-256c9c17-394a-4866-8579-f7dd257bf440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:11.401651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:11.551717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:11.553171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:11.635508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:11.702721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8eebc-7c15-493b-85c9-3d1cc6d3a8a4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:11.813673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:11.814211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:11.814488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:11.814761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:11.816408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:11.866999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:11.908279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:11.909875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7caa2-052d-47ee-a47c-037452321a41 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:11.927301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 709/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:12.720799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:12.721399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:12.721399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:12.722706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:12.727518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f10231-ba39-4bda-813d-1ea5c922a744 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:12.728072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f10231-ba39-4bda-813d-1ea5c922a744 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:12.728072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f10231-ba39-4bda-813d-1ea5c922a744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:12.803421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b7f10231-ba39-4bda-813d-1ea5c922a744 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:12.804628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b7f10231-ba39-4bda-813d-1ea5c922a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:12.898591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:13.048593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:13.050055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:13.098595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:13.194315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cad90-2e55-429f-b62b-9a73a9bd55b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:13.305624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:13.306316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:13.306601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:13.306876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:13.308554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:13.315761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:13.400893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:13.402485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec83e3e-072d-4d25-930b-611d4057be97 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:13.420540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 721/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:14.216170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:14.216684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:14.216684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:14.218043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:14.223026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0972-a2ce-4efd-8b0e-cbb9cdd3309d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:14.223446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0972-a2ce-4efd-8b0e-cbb9cdd3309d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:14.223584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0972-a2ce-4efd-8b0e-cbb9cdd3309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:14.298937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1d7c0972-a2ce-4efd-8b0e-cbb9cdd3309d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:14.300148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1d7c0972-a2ce-4efd-8b0e-cbb9cdd3309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:14.365982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:14.394982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:14.545905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:14.547356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:14.565227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:14.691857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd4ab9-bf32-4cf1-9372-a9b11e02e824 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:14.780430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:14.805926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:14.806473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:14.806753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:14.807026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:14.808736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:14.894920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:14.896539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ed44-b158-4076-b6e1-959d256865ec service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:14.914474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 712/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:15.713935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:15.714527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:15.714527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:15.716496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:15.721447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7fe8-df61-42ff-af95-b215a4df0227 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:15.721890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7fe8-df61-42ff-af95-b215a4df0227 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:15.722026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7fe8-df61-42ff-af95-b215a4df0227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:15.797566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f0fd7fe8-df61-42ff-af95-b215a4df0227 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:15.798759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f0fd7fe8-df61-42ff-af95-b215a4df0227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:15.841972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:15.895223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:16.046729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:16.048295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:16.199201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e59a8-871f-4258-8800-faf55cd14842 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:16.252938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:16.310812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:16.311682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:16.311996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:16.312272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:16.313893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:16.402519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:16.404724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43004281-3f70-4690-bc90-558b2d99ab19 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:16.422356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 724/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:17.227194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:17.227728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:17.227728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:17.229029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:17.234613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f803e-eaa7-4c61-a8e4-ee0f7af89d37 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:17.235067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f803e-eaa7-4c61-a8e4-ee0f7af89d37 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:17.235165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f803e-eaa7-4c61-a8e4-ee0f7af89d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:17.310726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6e6f803e-eaa7-4c61-a8e4-ee0f7af89d37 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:17.312227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6e6f803e-eaa7-4c61-a8e4-ee0f7af89d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:17.314215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:17.412139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:17.562221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:17.563652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:17.706314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc010-e233-4b79-98b3-477a8e4c91cf None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:17.732523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:17.817012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:17.817576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:17.817886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:17.818189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:17.819860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:17.915764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:17.917421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f888-fd20-47dc-a6ca-c32b036af3b0 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:17.934995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 715/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:19.819387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:19.819953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:19.819953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:19.821338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:19.827084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e570-5bb8-4432-87ac-661a0ba00287 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:19.827465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e570-5bb8-4432-87ac-661a0ba00287 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:19.827634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e570-5bb8-4432-87ac-661a0ba00287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:19.867414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:19.892807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-aeb4e570-5bb8-4432-87ac-661a0ba00287 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:19.894012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aeb4e570-5bb8-4432-87ac-661a0ba00287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:19.989281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:20.027613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:20.029253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:20.137600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01923f-35fc-4788-a5c9-2e417c8c9a84 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:20.169781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:20.211457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:20.212102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:20.212391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:20.212670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:20.214310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:20.326791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:20.328279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daad8c-de98-432b-b4d7-9588fee338ba service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:20.346990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 727/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:22:22.469942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:22.470488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:22.470488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:22.471823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:22.477107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab4a4-5c59-4080-9571-d1f8d2d5655c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:22.477544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab4a4-5c59-4080-9571-d1f8d2d5655c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:22.477695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab4a4-5c59-4080-9571-d1f8d2d5655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:22.542755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3e0ab4a4-5c59-4080-9571-d1f8d2d5655c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:22.543997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e0ab4a4-5c59-4080-9571-d1f8d2d5655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:22.556752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:22.640307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:22.678603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:22.680328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:22.782308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c312e-d0f7-4458-a2ed-e9868585ae3b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:22.794120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:22.854362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:22.854991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:22.855253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:22.855524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:22.857692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:22.963574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:22.965051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08053c36-0c1c-4827-b10f-5c9596631185 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:22.983241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 718/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:25.366318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:25.366838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:25.366932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:25.368287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:25.374200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c6c2-b601-42c4-bc24-7641f8cea7f9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:25.374656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c6c2-b601-42c4-bc24-7641f8cea7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:25.374763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4c6c2-b601-42c4-bc24-7641f8cea7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:25.423334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:25.444313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-96b4c6c2-b601-42c4-bc24-7641f8cea7f9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:25.445515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-96b4c6c2-b601-42c4-bc24-7641f8cea7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:25.539115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:22:25.577301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:25.578872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:25.681909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a140e-24de-4243-86d7-d6913cd04510 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:25.721702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:25.756675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:25.757282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:25.757561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:25.757833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:25.759472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:25.866019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:25.867426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dee38a-a28b-4d1f-b12f-23456e7ae4c7 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:25.884755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 730/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:27.980122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:27.980539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:27.980685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:27.982053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:27.987342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f72c7-e8e9-4d9c-9fbe-a52a9b530c41 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:27.987774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f72c7-e8e9-4d9c-9fbe-a52a9b530c41 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:27.987950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f72c7-e8e9-4d9c-9fbe-a52a9b530c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:28.054670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-427f72c7-e8e9-4d9c-9fbe-a52a9b530c41 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:28.055917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-427f72c7-e8e9-4d9c-9fbe-a52a9b530c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:28.077092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:28.152596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:28.191554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:28.193119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:28.296587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377efee-bbf2-45d6-baa2-5020f882a3a7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:28.316799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:28.369152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:28.369769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:28.370051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:28.370324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:28.371978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:28.478319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:28.479763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a134572-d693-44d2-af50-03b2dad9c653 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:28.498468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 721/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:30.553545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:30.554026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:30.554139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:30.555467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:30.561351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc97-4cde-4c46-af46-c4bf4e697a64 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:30.561758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc97-4cde-4c46-af46-c4bf4e697a64 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:30.561899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc97-4cde-4c46-af46-c4bf4e697a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:30.624375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:30.627348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eefccc97-4cde-4c46-af46-c4bf4e697a64 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:30.629130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eefccc97-4cde-4c46-af46-c4bf4e697a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:30.722566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:30.761344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:30.762933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:30.866452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cfad-6ed8-4811-b97d-3a5c0dab4d88 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:30.915397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:30.940614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:30.941220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:30.941498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:30.941770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:30.943372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:31.051037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:31.052422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2204391-76eb-4dc6-992b-8dfdc007707c service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:31.070127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 733/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:33.101579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:33.102130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:33.102130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:33.103490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:33.109170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4694-dd41-4489-90bb-b17c1ef95fbb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:33.109556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4694-dd41-4489-90bb-b17c1ef95fbb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:33.109712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4694-dd41-4489-90bb-b17c1ef95fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:33.175522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3b4e4694-dd41-4489-90bb-b17c1ef95fbb None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:33.176754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b4e4694-dd41-4489-90bb-b17c1ef95fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:33.207033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:33.394527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:33.434881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:33.436458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:33.543266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04926dc-7192-4b5a-881a-8fc912368004 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:33.573127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:33.618258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:33.618870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:33.619149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:33.619431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:33.621067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:33.727439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:33.728830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df4da3-c02c-4ac1-a877-4257f358869d service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:33.746641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 724/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:35.796999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:35.797570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:35.797570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:35.798871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:35.804003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4ca2-16b2-47f1-8f2c-187badf189d5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:35.804449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4ca2-16b2-47f1-8f2c-187badf189d5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:35.804615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4ca2-16b2-47f1-8f2c-187badf189d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:35.868945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ecba4ca2-16b2-47f1-8f2c-187badf189d5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:35.870164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ecba4ca2-16b2-47f1-8f2c-187badf189d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:35.878765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:35.966631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:36.004342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:36.005884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:36.109564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc295a43-b9be-4b57-b6c5-cf9e418866bf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:36.163177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:36.181623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:36.182227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:36.182507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:36.182778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:36.184398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:36.290275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:36.291692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ad979-2d7d-49e9-87ad-586d45fd0c45 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:36.309916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 736/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:38.425896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:38.426456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:38.426456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:38.427822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:38.433257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7676-61d2-440c-9731-cf8538c573be None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:38.433674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7676-61d2-440c-9731-cf8538c573be None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:38.433807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7676-61d2-440c-9731-cf8538c573be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:38.498900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-602e7676-61d2-440c-9731-cf8538c573be None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:38.500120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-602e7676-61d2-440c-9731-cf8538c573be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:38.537696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:38.596994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:38.634049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:38.635591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:38.740594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5c3b1-7dbd-42c0-8c46-17007ea2bfad None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:38.777823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:38.813910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:38.814531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:38.814812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:38.815087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:38.816767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:38.924789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:38.926226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da9469-5f1f-4d19-9d5b-4456e61b025b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:38.944505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 727/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:40.930365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:40.930946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:40.930946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:40.932342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:40.938315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca698-c954-45ee-801f-3c6b02411fe1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:40.938726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca698-c954-45ee-801f-3c6b02411fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:40.938875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcca698-c954-45ee-801f-3c6b02411fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:41.005853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fbcca698-c954-45ee-801f-3c6b02411fe1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:41.007067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fbcca698-c954-45ee-801f-3c6b02411fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:41.024693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:41.102048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:41.139135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:41.140730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:41.242331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca321f-7d15-4b75-921a-ae8757d9e03a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:41.310357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:41.317962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:41.318578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:41.318861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:41.319138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:41.320833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:41.427650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:41.429096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7473dc-e742-4689-a43b-db6a29476ede service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:41.446825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 739/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:43.452415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:43.453045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:43.453045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:43.454317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:22:43.459675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d34c4-34dd-40e6-a2d6-de19fc985c7f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:43.460144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d34c4-34dd-40e6-a2d6-de19fc985c7f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:43.460284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d34c4-34dd-40e6-a2d6-de19fc985c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:43.531732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cf8d34c4-34dd-40e6-a2d6-de19fc985c7f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:22:43.532990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cf8d34c4-34dd-40e6-a2d6-de19fc985c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:43.582463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:43.630574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:22:43.667120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:22:43.668713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:43.772355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb472b7-cfa7-4d6d-834b-13d6425f0f22 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:22:43.819337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:43.845731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:22:43.846378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:22:43.846658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:22:43.846924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:22:43.848567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:22:43.956232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:22:43.957670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f1df2-1417-4f77-863c-b51fd6bd95d2 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:22:43.975378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 730/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:22:53.628788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=22 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:22:53.973121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=66 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:03.394620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f21e4-5676-445a-b7e3-710d0464f150 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:03.395164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f21e4-5676-445a-b7e3-710d0464f150 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:03.395262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f21e4-5676-445a-b7e3-710d0464f150 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:03.396590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 741/1471] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:03.401852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7651b-4abf-4593-96ce-cb963026b5da None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:03.402290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7651b-4abf-4593-96ce-cb963026b5da None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:03.402395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7651b-4abf-4593-96ce-cb963026b5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:03.467672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9cd7651b-4abf-4593-96ce-cb963026b5da None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:03.469501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9cd7651b-4abf-4593-96ce-cb963026b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:03.565189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 731/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:03.648608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:03.648899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:03.649175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:03.650500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 742/1473] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 15:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:03.655242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395ff3e-b750-412c-ab71-69e8d727ffaf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:03.655576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395ff3e-b750-412c-ab71-69e8d727ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:03.655844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395ff3e-b750-412c-ab71-69e8d727ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:03.730925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6395ff3e-b750-412c-ab71-69e8d727ffaf None None] MFA Rules not processed for user `77f96ae3547b4be6bd6bb31c14903926`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:03.732198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6395ff3e-b750-412c-ab71-69e8d727ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:03.756841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:03.812403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 732/1474] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:03.820218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:03.821746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:03.925395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42a19e-53f5-410d-bb69-c835a3666928 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:03.997790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:03.998405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:03.998690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:03.998963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:04.000651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:04.054500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:04.110022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:04.111477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ae269-579e-43ba-bc74-4219b96f4aa1 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:04.129433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 743/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:06.190997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:06.191558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:06.191558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:06.192915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 733/1476] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:06.197982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb7a4-26b8-4230-896f-922771583d51 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:06.198398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb7a4-26b8-4230-896f-922771583d51 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:06.198567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb7a4-26b8-4230-896f-922771583d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:06.264426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4beb7a4-26b8-4230-896f-922771583d51 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:06.266154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4beb7a4-26b8-4230-896f-922771583d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:06.361380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 744/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:06.384276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:06.385788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:06.509687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd19d0-40d9-4794-b3c7-13d977fa87b8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:06.558477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:06.603890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:06.604508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:06.604786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:06.605055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:06.605750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:06.607502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:23:06.607502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:23:06.607502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:23:06.609181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0575d-1667-486e-8785-9b70ef2ee2ca admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:06.628258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 734/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 15:23:06 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:23:08.626404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:08.626971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:08.626971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:08.628349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 745/1479] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:08.633601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1d520-f170-433b-96cc-df581c95443d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:08.634033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1d520-f170-433b-96cc-df581c95443d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:08.634141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1d520-f170-433b-96cc-df581c95443d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:08.698264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-abe1d520-f170-433b-96cc-df581c95443d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:08.699456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-abe1d520-f170-433b-96cc-df581c95443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:08.793023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 735/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:08.835985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:08.837563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:08.847944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:08.984940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468ee2e-c72f-4c7e-8ae3-be70176237f2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:09.063780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:09.097191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:09.097827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:09.098104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:09.098373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:09.100024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:09.186159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:09.188285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd010e0d-2705-4b63-aeb3-1a5fed599284 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:09.206672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 746/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:11.399819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:11.400387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:11.400387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:11.401751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 736/1482] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:11.407456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3a9b7-24b5-4aaf-80c5-ebfdff717cc0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:11.407919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3a9b7-24b5-4aaf-80c5-ebfdff717cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:11.408042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3a9b7-24b5-4aaf-80c5-ebfdff717cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:11.472752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9ea3a9b7-24b5-4aaf-80c5-ebfdff717cc0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:11.473948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9ea3a9b7-24b5-4aaf-80c5-ebfdff717cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:11.519058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:11.569762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 747/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:23:11.606003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:11.607529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:11.632531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:11.711699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65365e70-6988-42db-bd34-b294bb80a7e7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:11.782518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:11.783125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:11.783404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:11.783674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:11.785313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:11.857636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:11.891556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:11.893042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275ace-c340-41e9-8066-8a12363ad34b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:11.912845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 737/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:13.977913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:13.978392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:13.978495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:13.979858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 748/1485] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:13.985574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904e56-20c1-4e69-be9b-9cd3b3a49d56 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:13.986008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904e56-20c1-4e69-be9b-9cd3b3a49d56 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:13.986123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904e56-20c1-4e69-be9b-9cd3b3a49d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:14.169939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fa904e56-20c1-4e69-be9b-9cd3b3a49d56 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:14.171153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fa904e56-20c1-4e69-be9b-9cd3b3a49d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:14.266389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 738/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:14.304674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:14.306229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:14.408731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e7f48-dfa2-48a3-9467-de5a087dc650 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:14.479326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:14.481197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:14.481894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:14.482189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:14.482669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:14.485205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:14.596952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:14.598302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1fcbb-15a5-4aa6-9840-171abcadd3a6 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:14.616784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 749/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:16.757399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:16.757978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:16.757978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:16.759283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 739/1488] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:23:16.764415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e802dc-3f60-4e7b-8377-68adadf30e11 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:16.764834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e802dc-3f60-4e7b-8377-68adadf30e11 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:16.764966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e802dc-3f60-4e7b-8377-68adadf30e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:16.831045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-35e802dc-3f60-4e7b-8377-68adadf30e11 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:16.832267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-35e802dc-3f60-4e7b-8377-68adadf30e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:16.883991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:16.927527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 750/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:16.968181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:16.969745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:17.007106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:17.076574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1859350-9148-4f88-95e6-d0e0e6725c26 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:17.149261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:17.149875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:17.150435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:17.150709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:17.152365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:17.230036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:17.261168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:17.262608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b6169-1cc9-417a-a740-ce2cfe6da2c9 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:17.280209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 740/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:19.474157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:19.474726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:19.474726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:19.476106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 751/1491] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 15:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:23:19.481364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9246-5de6-4cb2-8830-cc517cadf7b9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:19.481788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9246-5de6-4cb2-8830-cc517cadf7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:19.481896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9246-5de6-4cb2-8830-cc517cadf7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:19.545942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-23cd9246-5de6-4cb2-8830-cc517cadf7b9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:23:19.547059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-23cd9246-5de6-4cb2-8830-cc517cadf7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:19.648644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 741/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:23:19.687595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:23:19.689229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:19.794063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627996-ad33-49df-b6bd-b894d0ec1019 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:23:19.866355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:23:19.866963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:23:19.867244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:23:19.867518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:23:19.869176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:23:19.878679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:19.982507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:23:19.984481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348efe47-d04a-411a-b831-1a64279bb8bc service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:23:20.005534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 752/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:23:29.646740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=82 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:23:30.003181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=50 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:46.153589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:46.154088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:46.154202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:46.228133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:24:46.229284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:46.326369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 742/1494] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:24:46.335869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:46.335869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:46.335869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:46.413871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:24:46.415202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:46.518806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 753/1495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:24:46.523519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:46.524784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:46.536882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f5377-4185-467c-932c-5e594aefb56c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:46.598541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:46.631658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:46.632304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:46.632582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:46.632846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:46.633532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:24:46.633852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:46.634863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:46.634863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:24:46.634863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:24:46.635769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:46.647223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 743/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:24:46.655870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:46.665443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293443-56c8-40bc-8a3e-45b7dc8f6f07 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:46.725212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:46.759109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:46.759713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:46.760023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:46.760294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:46.760953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:46.762673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:46.762673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:24:46.762673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:24:46.763855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:46.820076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 754/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:24:46 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:24:46.824225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:46.825814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:46.860304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691c89-c422-40db-bd48-7d36f6bce3a5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:46.952175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:46.952791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:46.953071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:46.953890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:46.954557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:46.955578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:46.955578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:24:46.955578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:24:46.956803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:46.997520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 744/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:24:46 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:24:47.002156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:47.003667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.071564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedac9-d8a9-4d9e-8ac3-7b76d9222876 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:47.153877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:47.165224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:47.165839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.166119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.166394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.167081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:47.168116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:47.168116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:24:47.168116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:24:47.168998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:47.181113 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 755/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:24:47.185165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:47.186683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.199082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1209fe-bb6e-40f7-a414-1cb59d82545b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:47.223889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:47.293567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:47.294183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.294462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.294739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.295428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:47.296487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:47.296487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:24:47.296487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:24:47.297875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5036-d934-4b71-9ce8-35d9930dbf3b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:47.309933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 745/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:24:47.314757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:47.316319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.349271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fbbea-ee9f-48bb-8f3e-234bdf8c1f19 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:47.443528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:47.444222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.444500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.444775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] PATH_INFO: `/v3/projects/780aeaa8f12e4102a29685aa147e5729/users/dc8a766aee8947858077901fba7a2970/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.466592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] RBAC: Authorizing `identity:create_grant(project_id=780aeaa8f12e4102a29685aa147e5729, user_id=dc8a766aee8947858077901fba7a2970, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:47.467617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:47.467617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:24:47.467617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:24:47.469584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b924da-618f-4d61-8d3c-006073ea412f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:47.494168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 756/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:24:47 2022] PUT /identity/v3/projects/780aeaa8f12e4102a29685aa147e5729/users/dc8a766aee8947858077901fba7a2970/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:24:47.506949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3a1f-9ec6-4fa3-92ca-690fc938ab33 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.507244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3a1f-9ec6-4fa3-92ca-690fc938ab33 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.507527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3a1f-9ec6-4fa3-92ca-690fc938ab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.567033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ce1a3a1f-9ec6-4fa3-92ca-690fc938ab33 None None] MFA Rules not processed for user `dc8a766aee8947858077901fba7a2970`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:24:47.568522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ce1a3a1f-9ec6-4fa3-92ca-690fc938ab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.643739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 746/1502] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:24:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:24:47.657148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.657430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.657703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.659045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 757/1503] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 15:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:24:47.663616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e801-3343-457b-92de-eb3ff54effaf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.663941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e801-3343-457b-92de-eb3ff54effaf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:47.664219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e801-3343-457b-92de-eb3ff54effaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:47.695078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:47.739609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2bb1e801-3343-457b-92de-eb3ff54effaf None None] MFA Rules not processed for user `711efe02d15146dfb6a1a737abcee3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:24:47.740838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2bb1e801-3343-457b-92de-eb3ff54effaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.814001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 747/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 15:24:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:24:47.821900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:47.823409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:47.873043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:47.926326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409486-d96b-4753-897f-22d8a10dfbf8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:47.999016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:47.999629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:47.999947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:48.000222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:48.001846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:48.101547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:48.102970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:48.106490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:48.132309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 758/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:24:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:24:48.177163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081a514-29be-4757-9d6b-41c4adb0eff5 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:48.177398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081a514-29be-4757-9d6b-41c4adb0eff5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:48.177674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081a514-29be-4757-9d6b-41c4adb0eff5 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:48.178958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 748/1506] 10.222.0.36 () {52 vars in 830 bytes} [Wed Jul 27 15:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:24:48.182657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:48.184027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:48.282731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77686-00b1-4b59-b8d5-4d7f4599a4f7 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:48.373846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:48.374477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:48.374739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:48.375524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] PATH_INFO: `/v3/users/dc8a766aee8947858077901fba7a2970` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:48.387564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8a766aee8947858077901fba7a2970)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:48.388643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:48.388643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:24:48.388643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:24:48.389838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:48.399781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:48.452241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8a766aee8947858077901fba7a2970'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:48.452691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8a766aee8947858077901fba7a2970'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:48.462009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8a766aee8947858077901fba7a2970'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:48.474067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8a766aee8947858077901fba7a2970'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:48.491884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8a766aee8947858077901fba7a2970'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:48.497659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 759/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:24:48 2022] DELETE /identity/v3/users/dc8a766aee8947858077901fba7a2970 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:24:48.502643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:48.503115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:48.503230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:48.558889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:24:48.560105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:48.636379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:48.650143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 749/1508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:24:48.659707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:48.661001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:48.759822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab322a5-cf72-4742-9858-515f32b5db86 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:48.815436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:48.836884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:48.837499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:48.837778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:48.838053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:48.839692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:48.947764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:48.949204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6a3b9-b24f-40bc-afe0-3602e56b0829 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:48.968094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 760/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:24:49.302521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:24:49.303842 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:24:49.315948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda91e08-e5ce-406c-a910-5b512392b16d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:24:49.410902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:24:49.411545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:24:49.411852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:24:49.412133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] PATH_INFO: `/v3/projects/780aeaa8f12e4102a29685aa147e5729` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:24:49.421725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780aeaa8f12e4102a29685aa147e5729)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:24:49.423315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:24:49.423315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:24:49.423315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:24:49.424539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:24:49.480302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780aeaa8f12e4102a29685aa147e5729', 'request_id': 'req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:49.486897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780aeaa8f12e4102a29685aa147e5729', 'request_id': 'req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:49.497390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780aeaa8f12e4102a29685aa147e5729', 'request_id': 'req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:49.528490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] The token cache is being invalidate because project 780aeaa8f12e4102a29685aa147e5729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:24:49.529145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:49.529242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ba81fc3-e12c-4bda-ac5c-6be64aec0f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:24:49.530565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 750/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:24:49 2022] DELETE /identity/v3/projects/780aeaa8f12e4102a29685aa147e5729 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:24:58.965743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=62 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:24:59.526421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:26:28.792358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:28.792973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:28.792973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:28.794280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:28.798951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773ec0f-cd5a-40e0-a8e2-02f972ef8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:28.799356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773ec0f-cd5a-40e0-a8e2-02f972ef8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:28.799494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773ec0f-cd5a-40e0-a8e2-02f972ef8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:28.882303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9773ec0f-cd5a-40e0-a8e2-02f972ef8cc3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:28.883504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9773ec0f-cd5a-40e0-a8e2-02f972ef8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:28.987720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:29.145148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:29.147326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:29.264928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec2186-b837-4a40-9859-2de1a6ecdc86 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:29.270450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:29.271063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:29.271345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:29.271620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:29.279761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:29.281361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e5d89-7e94-4142-ae44-7f76a9176317 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:29.290754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 762/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:30.118357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7036c-03da-4259-9e84-e33a8d869f92 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:30.118893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7036c-03da-4259-9e84-e33a8d869f92 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:30.119025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7036c-03da-4259-9e84-e33a8d869f92 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:30.120334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:26:30.125135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735333f9-2ce5-436c-851a-f5def79fa654 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:30.125556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735333f9-2ce5-436c-851a-f5def79fa654 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:30.125678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735333f9-2ce5-436c-851a-f5def79fa654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:30.145516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-735333f9-2ce5-436c-851a-f5def79fa654 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:30.146699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-735333f9-2ce5-436c-851a-f5def79fa654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:30.154426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:30.305912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7036c-03da-4259-9e84-e33a8d869f92 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:30.321281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7036c-03da-4259-9e84-e33a8d869f92 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:30.328991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:30.329956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:30.330083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:30.330378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:30.340982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:30.342696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa17b11-bd79-4cef-ac5b-3c14dd7109bc service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:30.352411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 753/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:31.599939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90240-80a0-4214-a371-7cda262ed6a5 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:31.600510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90240-80a0-4214-a371-7cda262ed6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:31.600510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90240-80a0-4214-a371-7cda262ed6a5 None None] PATH_INFO: `/v3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:31.601774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 764/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Jul 27 15:26:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:26:31.611228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73514-0aa7-4bef-ac4d-2440f1b4db4e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:31.611665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73514-0aa7-4bef-ac4d-2440f1b4db4e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:31.611770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73514-0aa7-4bef-ac4d-2440f1b4db4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:31.670247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-14f73514-0aa7-4bef-ac4d-2440f1b4db4e None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:31.671456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-14f73514-0aa7-4bef-ac4d-2440f1b4db4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:31.688386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:26:31.743684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 754/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:26:35.685093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b530-58a7-47cd-9e0d-60f75229f983 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:35.685607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b530-58a7-47cd-9e0d-60f75229f983 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:35.685607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7b530-58a7-47cd-9e0d-60f75229f983 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:35.686898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 765/1519] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jul 27 15:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:35.691721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497757-e306-4cf2-b0f7-b43245e1fab8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:35.692182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497757-e306-4cf2-b0f7-b43245e1fab8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:35.692317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497757-e306-4cf2-b0f7-b43245e1fab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:35.711698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-de497757-e306-4cf2-b0f7-b43245e1fab8 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:35.712901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-de497757-e306-4cf2-b0f7-b43245e1fab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:35.721239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 755/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jul 27 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:35.729210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7b530-58a7-47cd-9e0d-60f75229f983 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:35.742766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7b530-58a7-47cd-9e0d-60f75229f983 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:35.747545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:35.748163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:35.748441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:35.748714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:35.756947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:35.758715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e218-2b08-414e-af2c-160265166b6e service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:35.768295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 766/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:26:41.577376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e74b-d048-492a-b325-7337e1408acd None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:41.577962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e74b-d048-492a-b325-7337e1408acd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:41.577962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e74b-d048-492a-b325-7337e1408acd None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:41.579264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:41.584848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a30d4-649f-49e7-b96e-9a03900554e3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:41.585322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a30d4-649f-49e7-b96e-9a03900554e3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:41.585424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a30d4-649f-49e7-b96e-9a03900554e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:41.604917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-432a30d4-649f-49e7-b96e-9a03900554e3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:41.606132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-432a30d4-649f-49e7-b96e-9a03900554e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:41.614184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:41.766532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e74b-d048-492a-b325-7337e1408acd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:41.781282 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e74b-d048-492a-b325-7337e1408acd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:41.787086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:41.787682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:41.787999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:41.788387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:41.796515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:41.798106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e969612-56d2-486f-b10b-bce04aeaabd9 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:41.808306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 757/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:26:42.620915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60604f01-9504-40e9-a118-6322b005202d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:42.621456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60604f01-9504-40e9-a118-6322b005202d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:42.621456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60604f01-9504-40e9-a118-6322b005202d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:42.622830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 15:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:42.627663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262add7-a3e3-4fcc-adec-eeb0aea5a796 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:42.628093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262add7-a3e3-4fcc-adec-eeb0aea5a796 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:42.628267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262add7-a3e3-4fcc-adec-eeb0aea5a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:42.647961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f262add7-a3e3-4fcc-adec-eeb0aea5a796 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:42.649160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f262add7-a3e3-4fcc-adec-eeb0aea5a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:42.656972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:42.806795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60604f01-9504-40e9-a118-6322b005202d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:42.822005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60604f01-9504-40e9-a118-6322b005202d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:42.827384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:42.828023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:42.828311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:42.828589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:42.836916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:42.838478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8ef6a-f1b2-4334-817e-da2c76936a26 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:42.847813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 769/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:46.724551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:46.725086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:46.725086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:46.726355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:46.731442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce332c2-62b2-42a6-82b8-4287407dbb05 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:46.731876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce332c2-62b2-42a6-82b8-4287407dbb05 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:46.732061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce332c2-62b2-42a6-82b8-4287407dbb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:46.751912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6ce332c2-62b2-42a6-82b8-4287407dbb05 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:46.753103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6ce332c2-62b2-42a6-82b8-4287407dbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:46.761794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:46.839621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:46.841461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:46.924459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e4db0-d5f3-45ae-bcbf-084e21277cd6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:46.928629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:46.929263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:46.929529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:46.929801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:46.939142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:46.940620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffedef-dcbe-4ce3-83bc-db384522c85b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:46.951257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 760/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:48.935276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cc6eb-8c28-4d8c-841b-f9d0ed3c2819 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:48.935957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cc6eb-8c28-4d8c-841b-f9d0ed3c2819 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:48.935957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cc6eb-8c28-4d8c-841b-f9d0ed3c2819 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:48.937217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:48.942940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc853-6df6-4621-8d66-4f4b64d1e08c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:48.943373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc853-6df6-4621-8d66-4f4b64d1e08c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:48.943480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc853-6df6-4621-8d66-4f4b64d1e08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:48.962729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8ccfc853-6df6-4621-8d66-4f4b64d1e08c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:48.963958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8ccfc853-6df6-4621-8d66-4f4b64d1e08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:48.971749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:49.007577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cc6eb-8c28-4d8c-841b-f9d0ed3c2819 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:49.018111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cc6eb-8c28-4d8c-841b-f9d0ed3c2819 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:49.021619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:49.022220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:49.022501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:49.022777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:49.031447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:49.032892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523a02a-1b10-4f4c-938e-7a5997fabdc3 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:49.042693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 772/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:52.456939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:52.457490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:52.457490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:52.458858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:52.464091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dddf5c-b350-4bb0-ab7a-60bbe7c233db None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:52.464520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dddf5c-b350-4bb0-ab7a-60bbe7c233db None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:52.464644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dddf5c-b350-4bb0-ab7a-60bbe7c233db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:52.483712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-01dddf5c-b350-4bb0-ab7a-60bbe7c233db None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:52.484861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-01dddf5c-b350-4bb0-ab7a-60bbe7c233db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:52.493103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:52.796307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:52.797967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:52.860866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:26:52.874032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3531-bacf-476a-a665-cac8c10a194c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:52.877536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:52.878144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:52.878449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:52.878723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:52.887851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:52.889284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016117f-5323-4ad6-9f95-16ac4c5cdd43 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:52.899067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 763/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:26:54.888802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:54.889377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:54.889377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:54.890724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 15:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:26:54.896627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd403f-918e-4875-9d88-90b07e18c493 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:54.897052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd403f-918e-4875-9d88-90b07e18c493 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:54.897179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd403f-918e-4875-9d88-90b07e18c493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:54.916719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2bcd403f-918e-4875-9d88-90b07e18c493 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:26:54.917906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2bcd403f-918e-4875-9d88-90b07e18c493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:54.925641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:26:55.225884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:26:55.227606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:26:55.240425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131904-b546-42f6-833d-a226b337bc9f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:26:55.243937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:26:55.244570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:26:55.244850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:26:55.245125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:26:55.254080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:26:55.256037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dac23-2dd8-4363-98af-6c845d18637c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:26:55.265293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 775/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:27:04.911369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=11 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:27:05.261203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:25.254591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf115-1d1a-4a36-9aaf-34c78d81ba5e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.255231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf115-1d1a-4a36-9aaf-34c78d81ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.255231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf115-1d1a-4a36-9aaf-34c78d81ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.279687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5ff5-29ec-4a88-96b1-86109dcc7a9c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.280319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5ff5-29ec-4a88-96b1-86109dcc7a9c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.280416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-099cf115-1d1a-4a36-9aaf-34c78d81ba5e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.280513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5ff5-29ec-4a88-96b1-86109dcc7a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.281273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-099cf115-1d1a-4a36-9aaf-34c78d81ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.297728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 765/1540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.311040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c97d5ff5-29ec-4a88-96b1-86109dcc7a9c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.312320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c97d5ff5-29ec-4a88-96b1-86109dcc7a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.312460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5336-5667-4a40-a00a-7db736b6a105 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.312691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5336-5667-4a40-a00a-7db736b6a105 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.312976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5336-5667-4a40-a00a-7db736b6a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.322147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 776/1541] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:25.328670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d14f2-7fa8-4111-b2fb-24995ab65de8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.328958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d14f2-7fa8-4111-b2fb-24995ab65de8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.329239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d14f2-7fa8-4111-b2fb-24995ab65de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.351679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-16fa5336-5667-4a40-a00a-7db736b6a105 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.353101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-16fa5336-5667-4a40-a00a-7db736b6a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.360086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fc0d14f2-7fa8-4111-b2fb-24995ab65de8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.361282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fc0d14f2-7fa8-4111-b2fb-24995ab65de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.369607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 766/1542] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 27 15:28:25.371076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.371371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.371650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.376807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 777/1543] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:25.378283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934deec-ddb6-45c6-b8f5-a7af4975f994 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.378568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934deec-ddb6-45c6-b8f5-a7af4975f994 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.378844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934deec-ddb6-45c6-b8f5-a7af4975f994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.403450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1934deec-ddb6-45c6-b8f5-a7af4975f994 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.403782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.404986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1934deec-ddb6-45c6-b8f5-a7af4975f994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.405113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.413262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 767/1544] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.413384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 778/1545] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.416269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.416408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6bedc-df24-4e94-9362-915c59852fdf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.416408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6bedc-df24-4e94-9362-915c59852fdf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.416408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6bedc-df24-4e94-9362-915c59852fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.430340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6d3a-7ba7-4abf-b3b0-73b493ae084b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.435108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.435534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3ec6bedc-df24-4e94-9362-915c59852fdf None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.435740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.436040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.436323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.436751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3ec6bedc-df24-4e94-9362-915c59852fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.437028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.437337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.438954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ae8de-c82c-4b85-8480-9b01e4894e80 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.446491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 779/1546] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.446977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bd22a-042b-4e77-8011-0bded589e505 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.447692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bd22a-042b-4e77-8011-0bded589e505 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.447692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bd22a-042b-4e77-8011-0bded589e505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.452837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 768/1547] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.453962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.454393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.454682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.471194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c85bd22a-042b-4e77-8011-0bded589e505 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.472651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c85bd22a-042b-4e77-8011-0bded589e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.475176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.476408 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.481145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 780/1548] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.482434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3028-f8b3-4cc7-96ca-60d369aea774 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.482718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3028-f8b3-4cc7-96ca-60d369aea774 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.482992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3028-f8b3-4cc7-96ca-60d369aea774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.488346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 769/1549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.490034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.503731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-21db3028-f8b3-4cc7-96ca-60d369aea774 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.504124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30addf37-3b02-4810-a01a-e8b354ea7859 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.505014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-21db3028-f8b3-4cc7-96ca-60d369aea774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.510666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.511280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.511560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.511863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.512560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.512862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.513755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 781/1550] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.515900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.516049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.516049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.516049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.526782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 770/1551] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.527493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.534899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.536124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.542815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bda64-ff5f-4c25-8997-3acc3798e57f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.544150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 782/1552] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.544988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.547367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.548219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.548504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.548777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.549484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.549787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.551399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f73a8-f181-42a4-a1c8-9bca5815e439 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.557378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b66ae-51e1-417d-8d6e-8e6309d33e38 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.562493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.563104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.563390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.563668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.564536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.564801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 771/1553] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.566015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbadb-64e8-4480-84b3-1a4105379840 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.566300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbadb-64e8-4480-84b3-1a4105379840 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.566404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fba37-c296-4716-9f6d-e7e99c8733e2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.566574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbbadb-64e8-4480-84b3-1a4105379840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.592454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-72dbbadb-64e8-4480-84b3-1a4105379840 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.593679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-72dbbadb-64e8-4480-84b3-1a4105379840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.602851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 772/1554] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.604085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb10086-e010-4be7-bc00-d655c1bc8ca5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.604369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb10086-e010-4be7-bc00-d655c1bc8ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.604646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb10086-e010-4be7-bc00-d655c1bc8ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.610105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 783/1555] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:25.611982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d226-50db-42a3-9680-b37743e79839 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.612283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d226-50db-42a3-9680-b37743e79839 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.612558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d226-50db-42a3-9680-b37743e79839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.625599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dbb10086-e010-4be7-bc00-d655c1bc8ca5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.626809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dbb10086-e010-4be7-bc00-d655c1bc8ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.632131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a554d226-50db-42a3-9680-b37743e79839 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.633283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a554d226-50db-42a3-9680-b37743e79839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.685422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 773/1556] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.686678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.686970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.687246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.696613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 784/1557] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.697445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d226-50db-42a3-9680-b37743e79839 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.707057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d226-50db-42a3-9680-b37743e79839 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.708559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.709796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.711658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.712296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.712601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.712880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.713575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.715473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.718191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 774/1558] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.719030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.729004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4115dd-b185-418b-8953-45ebdd5ad308 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.733977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.734570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.734847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.735233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.736156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.736496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.738388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.748969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 785/1559] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:25.749787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.751339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 775/1560] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.752606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.762468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a747d-c800-4a24-829a-6f4f7b8c5c76 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.818011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7236c-d995-49b3-a9f0-65a5c34952a6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.819670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.820325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.820610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.820887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.821592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.821897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.823523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.823716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.824397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.824720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.825003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.825692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.827634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.833059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:25.835667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 776/1561] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.836525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.842664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:25.849197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafbbf6-e8df-4dcd-915d-107d44a2b0f9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.854811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.855409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.855853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.856143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.856809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.860993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 786/1562] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:25.860993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.861227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdea3d-ca4a-4ed1-9468-970af4456940 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.880570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 777/1563] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:25.882717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.883007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.883280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.905730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:25.906944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:25.931252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae885-f32f-4cf5-987a-38a78328bec7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.935850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.936471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.936751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.937153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.937853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.938157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.939159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:25.939159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:28:25.939159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:28:25.940063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.953172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 787/1564] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.954022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.972146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 778/1565] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:25.972984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:25.973579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6716-bee9-47d2-beb3-4f884e3d9bf4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.979251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.979880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.980161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.980431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.981115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:25.981414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.982997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.983108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e333c-6c2b-42ab-a55d-b5bde41664ff None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:25.987896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:25.988501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:25.988784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:25.989058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:25.989717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:25.991666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:25.996021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 788/1566] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:25.996862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.010440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097032d-545a-44de-b834-b33079701d39 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.013733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 779/1567] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.014352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.017966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.018622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.018903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.019201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.019882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.021752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.024782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690acf25-5b12-4d2f-adb6-10b5a510e1f6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.029406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.030006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.030292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.030571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.031229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.032281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:26.032281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:28:26.032281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:28:26.033465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.054824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 789/1568] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:25 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.054824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.074937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 780/1569] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.076039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.120044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fd208-56a9-4572-aba1-859fdc73304b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.139216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097943d-a20a-4fbb-9c8f-5edf1fac1903 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.143743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.144389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.144672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.144948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.145639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.147804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.158051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 781/1570] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.158959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.168618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.169048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd4f6-3358-4205-8757-b57266ac6821 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.173654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.174288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.174567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.174844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.175520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.177442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.178630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.179255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.179543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.179822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.180520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.181520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:26.181520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:28:26.181520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:28:26.182719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.204842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 790/1571] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.205836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.206236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 782/1572] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.207080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.216295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507b2e4-04df-412c-8a01-df9bfc7db20b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.229860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.271536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be94c-e008-4f9d-be94-1e1ae0e36961 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.273638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.274260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.274542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.274821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.275511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:26.275818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.276700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.277313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.277447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.277596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.277878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.278535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.280512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.291318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 791/1573] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:26.292368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.307438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1992d2-8153-4ea6-9913-c822259e794d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.307665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 783/1574] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.308471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.366096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.366705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.366996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.367272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.367990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.369790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.370573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab107-a5f2-4f2e-80bf-600f220e485e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.375202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.375866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.376146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.376423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.377112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.379021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.379633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 792/1575] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.380461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.390502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a66c-e5b8-453c-8e57-0d75b8327d46 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.395688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.396326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.396610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.396888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.398115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.400036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.401189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 784/1576] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.402012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.411669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08f1f-983d-4694-849b-6eff2a91c093 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.417421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.418158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.418440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.418716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.419401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.420718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 793/1577] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.423778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.424037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.431543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974349-1230-4bf8-a693-901d0466cfef admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.431979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 785/1578] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.432676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.437692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.438283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.438560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.438832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.439510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.441116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.443768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96e8a-1303-4430-8ffb-9c01f7961256 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.449152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.449736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.450021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.450299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.450942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.453081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 794/1579] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.453218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.453315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.466552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a596e5-5936-42c1-9a76-2fde8e21daa1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.471632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.472293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.472862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.473144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.473801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.475754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.476016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 786/1580] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.476947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.486241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfa7da-40a2-428c-8a3b-68a81d02924f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.490937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.491528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.491814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.492116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.493029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.495199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.505313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 795/1581] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.505313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.520257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 787/1582] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.520965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.529776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.532607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efb4f3-e3fe-4734-894c-8701ad9efc56 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.566563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ea8a-00c7-4ab3-8bb7-4dd4b897541e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.571952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.572842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.572842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.573130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.573815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.575405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.577091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.586257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 796/1583] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.587055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.592338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.592961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.593245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.593520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.594226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.595881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.596442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac072afe-8b8e-4914-8fbb-a76c1a0d50ef admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.600989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.601579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.601860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.602131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.602794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.604374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.608948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 788/1584] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.608948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.617578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 797/1585] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.617905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816dce-dd2a-40db-b513-42530aa2a705 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.618370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.622509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.623159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.623439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.623716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] PATH_INFO: `/v3/projects/225967f646634875852c13d075dcb1fc/users/86674559f1a94a17925779fe137b2639/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.627538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4eb0-7d87-49af-8ee6-9a9f3067a999 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.632505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.633083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.633364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.633640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.634306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.635890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.639786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] RBAC: Authorizing `identity:create_grant(project_id=225967f646634875852c13d075dcb1fc, user_id=86674559f1a94a17925779fe137b2639, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.640851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:26.640851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:28:26.640851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:28:26.642759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.650042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 798/1586] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.650042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.656969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 789/1587] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:26 2022] PUT /identity/v3/projects/225967f646634875852c13d075dcb1fc/users/86674559f1a94a17925779fe137b2639/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:26.657754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.657901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c2e39-3d33-4840-9df2-6be706a066ec admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.714822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.715442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.715725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.716029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.716721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.718304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.724418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625a71d-c8ca-4c01-8c2f-d5eb3df3069d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.728321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 799/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.729120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.731989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.732639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.732921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.733221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.733894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.735821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.738917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24db1c-fcf3-443c-a538-0e68ad08e3ee admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.743981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.744574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.744856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.745131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.745807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.747530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.758772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 790/1589] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:26.759486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 800/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.759616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.760570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.769734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea133e4-2b20-422e-b174-8a59883502e2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.769922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3684-3d95-4afc-a19c-6860abb0d1a6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.774419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.774566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.775011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.775126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.775294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.775392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.775576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.775719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] PATH_INFO: `/v3/projects/736d8803a13e4dddb1ef32b7ab11eb1c/users/51657a0e4865469999bdf8cd5102dad1/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.776286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.777852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.787644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 801/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.788435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.790151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=736d8803a13e4dddb1ef32b7ab11eb1c, user_id=51657a0e4865469999bdf8cd5102dad1, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.792924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.798111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed98718-2399-461e-9c35-51c2efd89b95 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.804204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.804949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.805249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.805640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.806330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.807544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 791/1592] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:26 2022] PUT /identity/v3/projects/736d8803a13e4dddb1ef32b7ab11eb1c/users/51657a0e4865469999bdf8cd5102dad1/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:26.807942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485fd77-837d-4cb4-ad4f-e3bd08051bb3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.808360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.818236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 802/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:26.820484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.820769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.821045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.848177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] MFA Rules not processed for user `86674559f1a94a17925779fe137b2639`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:26.849400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:26.874090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ee17-1bf3-4f58-9c9e-8c351d7d4abf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.878702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.879364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.879641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.879950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] PATH_INFO: `/v3/projects/e1c7c86e4b1c40ec8e4335a2770353f5/users/1989ddc458d645af8d017987a6b93809/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.894770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c7c86e4b1c40ec8e4335a2770353f5, user_id=1989ddc458d645af8d017987a6b93809, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.897949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.898182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.910681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 792/1594] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:26 2022] PUT /identity/v3/projects/e1c7c86e4b1c40ec8e4335a2770353f5/users/1989ddc458d645af8d017987a6b93809/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:26.911658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.920479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 803/1595] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:26.921316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:26.924972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:26.935671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9fa01-5ff8-4307-82f5-151747b8c9d1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.985697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34e6e1-b238-4985-95cb-5c12c149abe2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:26.990348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.990990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.991268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.991540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.992267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:26.993866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:26.997448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:26.998068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:26.998350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:26.998628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:26.999319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.000947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.003798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 793/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:27.004638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.012106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 804/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:27.012106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.014761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c85e4-072f-4e64-b2c3-efc34301970e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.019368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.019980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.020260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.020527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.021195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.021397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c971-a153-4c31-abdb-6062d2de44a0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.023315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186b956-56a5-4dc0-9c6f-f63fcc74caa3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.027036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.027798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.028244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.028982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] PATH_INFO: `/v3/projects/cfd56d6b673a4a74943ba094ae3f6cda/users/ac794ca0ac284d2bb417a07239f34e55/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.033205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 794/1598] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:27.034605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461f6c-02a4-4537-9b8e-41c3af613cef None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.034994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461f6c-02a4-4537-9b8e-41c3af613cef None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.035371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461f6c-02a4-4537-9b8e-41c3af613cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.044822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd56d6b673a4a74943ba094ae3f6cda, user_id=ac794ca0ac284d2bb417a07239f34e55, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.047529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.054414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f4461f6c-02a4-4537-9b8e-41c3af613cef None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.055751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f4461f6c-02a4-4537-9b8e-41c3af613cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.061720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 805/1599] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:27 2022] PUT /identity/v3/projects/cfd56d6b673a4a74943ba094ae3f6cda/users/ac794ca0ac284d2bb417a07239f34e55/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:27.062722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.116294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 795/1600] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.117701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.118005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.118258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.125965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b935b-d289-4906-a08c-09ce2390af5b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.130531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.131237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.132130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.132444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] PATH_INFO: `/v3/projects/fddbf2261d1f42d0a41aa921197fc181/users/a2b027a2c8dd4459b1309faab28f3d4b/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.138091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.139277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.146978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 796/1601] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.148285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.151680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddbf2261d1f42d0a41aa921197fc181, user_id=a2b027a2c8dd4459b1309faab28f3d4b, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.154372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.157441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46a0b6-23f8-459c-bdd8-da1fcf19081b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.163240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.163920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.164206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.164483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.165162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.167050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.168603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 806/1602] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:27 2022] PUT /identity/v3/projects/fddbf2261d1f42d0a41aa921197fc181/users/a2b027a2c8dd4459b1309faab28f3d4b/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:27.169457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.202477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 797/1603] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:27.203417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.221522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:27.239277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed0d3-2541-4047-a9a4-1386ff53283a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.265541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fe41e-35ca-47f9-96c0-c167da4c0156 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.270126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.270792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.271074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.271349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] PATH_INFO: `/v3/projects/85b0f788133b43dba1ddcb9be9d9c6f7/users/571667712bee4f87a22fff9490151b5a/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.282564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:27.286520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b0f788133b43dba1ddcb9be9d9c6f7, user_id=571667712bee4f87a22fff9490151b5a, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.289513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.296862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.297587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.297845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.298310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] PATH_INFO: `/v3/projects/ee338fc85c214e2da3127c44b710cefe/users/4593106ecf604fe891b7a17c21141c24/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.303400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 798/1604] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:27 2022] PUT /identity/v3/projects/85b0f788133b43dba1ddcb9be9d9c6f7/users/571667712bee4f87a22fff9490151b5a/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:27.304513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.320712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee338fc85c214e2da3127c44b710cefe, user_id=4593106ecf604fe891b7a17c21141c24, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.323401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fda37-110d-4f8c-8b45-81332b9544f3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.336934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 807/1605] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:27 2022] PUT /identity/v3/projects/ee338fc85c214e2da3127c44b710cefe/users/4593106ecf604fe891b7a17c21141c24/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:27.338157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.338439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.338710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.368069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115214d-7e3e-4c3c-a363-69ec6387dbd0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.369299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] MFA Rules not processed for user `ac794ca0ac284d2bb417a07239f34e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.370519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.442803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.443533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.443855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.444729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.445436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.447064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.456457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 808/1606] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.456457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.465227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 799/1607] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:27.466232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.520992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a9492-b7b5-48c2-87f8-60927c691bae None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.525044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.525659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.525942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.526212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.533421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f39c5-1c83-4362-a373-98931c338537 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.535006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.536455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611a570-769c-4bf7-a4a1-8bcca68a0c7d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.537132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.537805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.538090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.538369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.546138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 809/1608] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:27.547565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.547717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20f55-41d5-418f-a7bd-1071e4d01433 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.547852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20f55-41d5-418f-a7bd-1071e4d01433 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.548175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20f55-41d5-418f-a7bd-1071e4d01433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.549026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.561955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 800/1609] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:27.562161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:27.563432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.568510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3fc20f55-41d5-418f-a7bd-1071e4d01433 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.571141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3fc20f55-41d5-418f-a7bd-1071e4d01433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.575676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896681f-36c4-4685-9998-caca3393285b service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.579557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 810/1610] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.580892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.581250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.581595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.582467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.583552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.583662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.584028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.584792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.586964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900e71d-8310-4291-b7fa-db517714e405 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.617888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 801/1611] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:27.618125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] MFA Rules not processed for user `571667712bee4f87a22fff9490151b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.618231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556dc3-4f60-496d-835a-d30f838e9362 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.618231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556dc3-4f60-496d-835a-d30f838e9362 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.618231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556dc3-4f60-496d-835a-d30f838e9362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.618495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.648752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4f556dc3-4f60-496d-835a-d30f838e9362 None None] MFA Rules not processed for user `4593106ecf604fe891b7a17c21141c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.650221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4f556dc3-4f60-496d-835a-d30f838e9362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.656247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:27.692191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 811/1612] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.693104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.705254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d944c6-acfd-43c2-ba5b-e685c09175c1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.710268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.711320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.711706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.712119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] PATH_INFO: `/v3/projects/ddfdba8b75e643b485b2237ae070a1b2/users/7a964a003f8e4f1da603a13eb7f534ca/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.730425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfdba8b75e643b485b2237ae070a1b2, user_id=7a964a003f8e4f1da603a13eb7f534ca, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.733977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.738348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 802/1613] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:27.738348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.738348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.738348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.739715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 803/1614] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 15:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:27.741535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.743976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.752238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 812/1615] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:27 2022] PUT /identity/v3/projects/ddfdba8b75e643b485b2237ae070a1b2/users/7a964a003f8e4f1da603a13eb7f534ca/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:27.757739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.813366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065b98-d110-4317-9716-4b76fc878d82 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.817130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.817754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.818034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.818307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.832218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d232-e7dd-4f53-82b1-3aaae4927c1b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.840111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.841100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.841471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.842063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.843248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.845345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3dffb-8296-456f-b515-213ee986bb7a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.864203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 813/1616] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:27.864203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b757-1c23-4bab-874f-9da37b72e018 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.864203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b757-1c23-4bab-874f-9da37b72e018 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.864203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b757-1c23-4bab-874f-9da37b72e018 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.865652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 814/1617] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 15:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:27.867300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7b757-1c23-4bab-874f-9da37b72e018 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:27.883275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7b757-1c23-4bab-874f-9da37b72e018 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:27.888635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:27.889480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.889872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.890330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.891155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.892332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:27.893308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc80e90-a12c-4994-be03-b139a1597d09 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.893931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fe518-5027-4a87-89b2-4f7174fa65eb service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:27.904665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 804/1618] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:27.905841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.906203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.906592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.950940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 815/1619] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:27.953078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945c97-08b9-4128-a8b6-828f587704d2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:27.953536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945c97-08b9-4128-a8b6-828f587704d2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:27.953809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945c97-08b9-4128-a8b6-828f587704d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:27.955364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.957459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:27.979943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0f945c97-08b9-4128-a8b6-828f587704d2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:27.981201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0f945c97-08b9-4128-a8b6-828f587704d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:28.011386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:28.044527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:28.062745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 805/1620] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:28.065693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.080329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 816/1621] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:28.082146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.082146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.082348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.107397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:28.108845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:28.128561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 817/1622] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:28.129870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.143429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4552-6b1b-4960-bbaf-9624dbf8af14 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.163227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.164206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.164966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.165363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.166443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.168873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.176574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc599-ef3b-486e-85b5-00deced17fe2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.182913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.184242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.184606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.185071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] PATH_INFO: `/v3/projects/eb9c821239e24aef9bfdc157a2f1301a/users/a31b3ecfbef54ab6ad4e17cda4f29051/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.202546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 818/1623] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:28 2022] POST /identity/v3/users => generated 296 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:28.202546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.206322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9c821239e24aef9bfdc157a2f1301a, user_id=a31b3ecfbef54ab6ad4e17cda4f29051, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.209344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.219760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928e9a7-cf7c-4f78-a474-798366698d68 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.228208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.229171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.229553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.229925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.230986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 806/1624] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:28 2022] PUT /identity/v3/projects/eb9c821239e24aef9bfdc157a2f1301a/users/a31b3ecfbef54ab6ad4e17cda4f29051/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:28.239171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.309067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.311332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.314306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6499e-9d42-4956-9a5b-f766b514ec26 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.319113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.320103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.320548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.320774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.328932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 819/1625] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:28.329969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.345122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dc74-6537-4bfc-b196-afae53a7358b service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.422283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.424173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.430249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.430941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.431418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.431701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.439028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 807/1626] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:28.440384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.452455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3516b8-6a53-4e6a-b80c-64c806f0a4e4 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.458650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.460111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.460784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.461432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.462300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.464297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.468186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:28.479092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 808/1627] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:28.480858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.494679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374fa1e-6153-442e-bd54-49ea2ab50a32 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.497750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:28.501773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.502924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.503392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.504312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.505888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.508902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.512612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.519297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.526306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 809/1628] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:28.526306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.537160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43317b-7cfc-4896-ac98-b74377e54505 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.544070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 820/1629] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:28.544944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.556432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.570061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aca5-cd4e-488e-80c9-6b1030acc49c service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.570966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.575618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.576041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] PATH_INFO: `/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea/users/ce9f4146e0ca471a882ccf9b99c75a57/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.583273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.584007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.585630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.585839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] PATH_INFO: `/v3/projects/eb9c821239e24aef9bfdc157a2f1301a/users/a31b3ecfbef54ab6ad4e17cda4f29051/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.590763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9c821239e24aef9bfdc157a2f1301a, user_id=a31b3ecfbef54ab6ad4e17cda4f29051, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.596940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3453541-51f3-44c1-88d4-8a84cab5d296 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.626071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 821/1630] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:28 2022] PUT /identity/v3/projects/eb9c821239e24aef9bfdc157a2f1301a/users/a31b3ecfbef54ab6ad4e17cda4f29051/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:28.628242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33bfd34b2cb49b4ae969afb9abb68ea, user_id=ce9f4146e0ca471a882ccf9b99c75a57, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.634405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.637123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab84e-ff86-4b0e-a354-206c49afdc4b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.637382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab84e-ff86-4b0e-a354-206c49afdc4b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.637724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab84e-ff86-4b0e-a354-206c49afdc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.667416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 810/1631] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:28 2022] PUT /identity/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea/users/ce9f4146e0ca471a882ccf9b99c75a57/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:28.669036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.703037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b57ab84e-ff86-4b0e-a354-206c49afdc4b None None] MFA Rules not processed for user `a31b3ecfbef54ab6ad4e17cda4f29051`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:28.707047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b57ab84e-ff86-4b0e-a354-206c49afdc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:28.826781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadabd7b-b0a8-424f-b536-fde529e90560 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.843403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.844221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.844601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.844975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.846260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.848138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.871181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 822/1632] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:28.876663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 811/1633] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:28.878111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca195-203c-4cb3-aca5-5256f747acac None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.878562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca195-203c-4cb3-aca5-5256f747acac None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.879017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca195-203c-4cb3-aca5-5256f747acac None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.880120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.880815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 823/1634] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 15:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:28:28.885895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fca195-203c-4cb3-aca5-5256f747acac None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:28.930441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600533f-0b91-48f5-b511-a0a220ce2b42 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:28.942829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:28.943520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.943802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.944904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] PATH_INFO: `/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea/users/ce9f4146e0ca471a882ccf9b99c75a57/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.964775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33bfd34b2cb49b4ae969afb9abb68ea, user_id=ce9f4146e0ca471a882ccf9b99c75a57, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:28.967651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac85fa-8a2e-410a-9541-0ceb265b3ed2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:28.996018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 812/1635] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:28 2022] PUT /identity/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea/users/ce9f4146e0ca471a882ccf9b99c75a57/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:28.996018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5fb9a-e702-436b-8d9b-3ac388f2a4e6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:28.996018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5fb9a-e702-436b-8d9b-3ac388f2a4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:28.996018 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5fb9a-e702-436b-8d9b-3ac388f2a4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:28.997616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fca195-203c-4cb3-aca5-5256f747acac None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:29.033632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2aa5fb9a-e702-436b-8d9b-3ac388f2a4e6 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:29.034818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2aa5fb9a-e702-436b-8d9b-3ac388f2a4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:29.148145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:29.148783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.149071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.149335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.195784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 813/1636] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:29.203044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85a78-0db3-4a3f-88d3-822a313ff458 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.203430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85a78-0db3-4a3f-88d3-822a313ff458 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.203622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85a78-0db3-4a3f-88d3-822a313ff458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.229790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:29.266577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7fe85a78-0db3-4a3f-88d3-822a313ff458 None None] MFA Rules not processed for user `ce9f4146e0ca471a882ccf9b99c75a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:29.267816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7fe85a78-0db3-4a3f-88d3-822a313ff458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:29.289519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:29.292036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5ece4-750a-40f3-8836-049ef6153bdc service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:29.317624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:29.330187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 824/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:28:29.337738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469e73-9c2f-4cbe-ac4a-506cc2d491da None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.339527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469e73-9c2f-4cbe-ac4a-506cc2d491da None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.340037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469e73-9c2f-4cbe-ac4a-506cc2d491da None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.346519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 825/1638] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 15:28:29 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:29.349748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f469e73-9c2f-4cbe-ac4a-506cc2d491da None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:29.391253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f469e73-9c2f-4cbe-ac4a-506cc2d491da None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:29.398272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:29.400032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.401201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.401644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.417880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 814/1639] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:29.417880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.417880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.417880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.456133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:29.457354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:29.466662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:29.474253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:29.476079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 815/1640] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:29.509799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:29.525079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bdcd8-113d-491c-8238-75c3448374e7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:29.530529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:29.531163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.531443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.532500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.537233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 826/1641] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:29.558322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:29.559793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:29.578965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 816/1642] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:28:29.677862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:29.697428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a840d-307f-4cf1-ad82-c884fb96722c service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:29.707047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:29.710565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:29.712140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:29.712826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:29.766503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:29.768493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:29.804235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 827/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:30.860479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:30.873511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b483e3b-8507-4ff8-a856-3b31038ef2e8 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:30.883417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:30.884442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:30.884908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:30.885400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:31.008377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:31.010143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953136f-58e6-4246-863e-a69e60cbf8d5 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:31.030527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 817/1644] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:32.041142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:32.179635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e5a-756b-413e-b159-be9674628221 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:32.187749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:32.188448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:32.188736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:32.189017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] PATH_INFO: `/v3/users/86674559f1a94a17925779fe137b2639` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:32.200056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86674559f1a94a17925779fe137b2639)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:32.202311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:32.341030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86674559f1a94a17925779fe137b2639'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:32.342131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86674559f1a94a17925779fe137b2639'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:32.361727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86674559f1a94a17925779fe137b2639'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:32.382979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86674559f1a94a17925779fe137b2639'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:32.421063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86674559f1a94a17925779fe137b2639'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:32.435326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 828/1645] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:32 2022] DELETE /identity/v3/users/86674559f1a94a17925779fe137b2639 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:32.441028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:32.441525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:32.441722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:32.504932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:32.507618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:32.623077 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 818/1646] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:32.640427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:32.642723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:32.714017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:32.858153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8f62-a62b-4ba0-bed2-3496b8f88be6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:32.864378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:32.865823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:32.866180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:32.866418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:32.904943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:32.906415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:32.949122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 829/1647] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:34.640371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:34.643248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:34.674812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c04da-deaf-4310-bee8-3733a8cbe450 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:34.698026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:34.699673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:34.703856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:34.704388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] PATH_INFO: `/v3/projects/225967f646634875852c13d075dcb1fc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:34.707982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=225967f646634875852c13d075dcb1fc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:34.712774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:34.818948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:34.853405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225967f646634875852c13d075dcb1fc', 'request_id': 'req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:34.861488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225967f646634875852c13d075dcb1fc', 'request_id': 'req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:34.879106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225967f646634875852c13d075dcb1fc', 'request_id': 'req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:34.898321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] The token cache is being invalidate because project 225967f646634875852c13d075dcb1fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:34.898687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:34.899692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3bfd1ca8-33cd-43ba-a7df-0ab67274b052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:34.912334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 819/1648] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:34 2022] DELETE /identity/v3/projects/225967f646634875852c13d075dcb1fc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:34.920727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:34.936656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81642f-b1e7-4ad5-a3e8-592149081215 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:35.157260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:35.157260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:35.157558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:35.157833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] PATH_INFO: `/v3/users/a31b3ecfbef54ab6ad4e17cda4f29051` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:35.167036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31b3ecfbef54ab6ad4e17cda4f29051)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:35.169203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:35.294809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31b3ecfbef54ab6ad4e17cda4f29051'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:35.295495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31b3ecfbef54ab6ad4e17cda4f29051'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:35.308607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31b3ecfbef54ab6ad4e17cda4f29051'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:35.323666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31b3ecfbef54ab6ad4e17cda4f29051'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:35.342540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-407f763f-0fef-44b4-9f49-7a45baf466b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31b3ecfbef54ab6ad4e17cda4f29051'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:35.353510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 830/1649] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:34 2022] DELETE /identity/v3/users/a31b3ecfbef54ab6ad4e17cda4f29051 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:35.776820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:35.781114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:35.782253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:35.877110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] MFA Rules not processed for user `51657a0e4865469999bdf8cd5102dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:35.878655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:36.022396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 820/1650] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:36.032613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960250d6-46fb-4dfb-8ab3-8d98fddd1c63 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:36.033136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960250d6-46fb-4dfb-8ab3-8d98fddd1c63 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:36.033205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960250d6-46fb-4dfb-8ab3-8d98fddd1c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:36.057665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-960250d6-46fb-4dfb-8ab3-8d98fddd1c63 None None] MFA Rules not processed for user `51657a0e4865469999bdf8cd5102dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:36.058987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-960250d6-46fb-4dfb-8ab3-8d98fddd1c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:36.071264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 831/1651] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:36.082602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:36.085835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:36.230095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e469e58-1f34-439a-aa92-2d0d99235c2c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:36.240183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:36.241425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:36.241705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:36.241981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:36.271123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:36.273166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddedf36-fb38-43b9-90b4-066036c515d4 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:36.285520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 821/1652] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:37.348648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:37.349133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:37.349509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:37.351585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 832/1653] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 15:28:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:37.357294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:37.358250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:37.358736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:37.374141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:37.376223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:37.502872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:37.506949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:37.529961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:37.575459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:37.618838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3c53-14c3-451b-917e-b9991a784e2e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:37.632830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:37.633549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:37.633826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:37.634099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] PATH_INFO: `/v3/users/ac794ca0ac284d2bb417a07239f34e55` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:37.638360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac794ca0ac284d2bb417a07239f34e55)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:37.643344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:37.676909 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 822/1654] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:37.693367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:37.696986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:37.730383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00612b0-eef8-4d74-896b-ad39855e80bb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:37.745022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:37.745684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:37.745944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:37.746598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:37.769610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:37.771336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee1753-97b9-4df4-b7f3-ca34c3f425b8 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:37.802349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 823/1655] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:37.809910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac794ca0ac284d2bb417a07239f34e55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:37.811072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac794ca0ac284d2bb417a07239f34e55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:37.834378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac794ca0ac284d2bb417a07239f34e55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:37.848433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac794ca0ac284d2bb417a07239f34e55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:37.882430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac794ca0ac284d2bb417a07239f34e55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:37.894606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 833/1656] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:37 2022] DELETE /identity/v3/users/ac794ca0ac284d2bb417a07239f34e55 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:28:37.899545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:37.899797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:37.900358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:37.945821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:37.947212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:37.986368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:37.988264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:38.065870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 824/1657] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:38.083306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:38.085678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:38.133629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7f56-6995-493d-a5f3-78b246a39aab admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:38.139000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:38.141507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.141928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.143459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] PATH_INFO: `/v3/projects/eb9c821239e24aef9bfdc157a2f1301a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.144994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9c821239e24aef9bfdc157a2f1301a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:38.147396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:38.147396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 27 15:28:38.147396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: Jul 27 15:28:38.149658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:38.199172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab4005-5106-4918-a5c9-ce59e5703278 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:38.211217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:38.212125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.212809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.213325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.243578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:38.246576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441b38-e61f-4ecd-b337-1768a3f1b626 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:38.297585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 825/1658] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:38.310549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9c821239e24aef9bfdc157a2f1301a', 'request_id': 'req-a036c955-9df2-48e6-81a1-88f279c05955'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:38.328291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9c821239e24aef9bfdc157a2f1301a', 'request_id': 'req-a036c955-9df2-48e6-81a1-88f279c05955'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:38.332703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:38.341612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9c821239e24aef9bfdc157a2f1301a', 'request_id': 'req-a036c955-9df2-48e6-81a1-88f279c05955'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:38.361387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] The token cache is being invalidate because project eb9c821239e24aef9bfdc157a2f1301a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:38.361948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:38.362626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a036c955-9df2-48e6-81a1-88f279c05955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:38.367545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 834/1659] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:37 2022] DELETE /identity/v3/projects/eb9c821239e24aef9bfdc157a2f1301a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:38.369620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.370361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.370650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.406439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:38.408154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:38.547784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 826/1660] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:38.550855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.551337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.551690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.597639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:38.599402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:38.644032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 835/1661] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:38.645414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:38.662854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8cd9a-c398-48f0-88b0-38156a424c20 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:38.679177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:38.680191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.680666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.681131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.682067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:38.682563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:38.684832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:38.687587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:38.719155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 827/1662] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:38.724922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:38.749015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d6905-d027-4c8f-8fd2-00f9d41b7001 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:38.784369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:38.787531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:38.789484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:38.792477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:38.804653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:38.810508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:38.913928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 836/1663] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:38 2022] POST /identity/v3/projects => generated 363 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:38.918182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:38.937092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5d3d9-bd0b-46bc-a5e6-85d56144e7ce admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.083278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:39.083944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.084295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.084569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.085249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:39.094501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:39.138881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:39.140792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.174116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e6464-531a-443a-a4c7-e8baadf154a6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.178575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 828/1664] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:38 2022] POST /identity/v3/users => generated 282 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:39.179532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:39.193742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:39.194736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.198020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.200923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] PATH_INFO: `/v3/projects/cfd56d6b673a4a74943ba094ae3f6cda` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.202487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd56d6b673a4a74943ba094ae3f6cda)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:39.204479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:39.205724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf607ad-fcdd-4c10-bb10-300dfc215775 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.232699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:39.233287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.233570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.233848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.234555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:39.239395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b3c31-1448-4c84-9825-1afc6bcb48a4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:39.293068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 829/1665] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:39.294574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337e708-d080-4bea-bcb9-32f5b968d786 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.294938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337e708-d080-4bea-bcb9-32f5b968d786 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.295170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337e708-d080-4bea-bcb9-32f5b968d786 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.296628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 830/1666] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jul 27 15:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:39.305513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337e708-d080-4bea-bcb9-32f5b968d786 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:39.308263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd56d6b673a4a74943ba094ae3f6cda', 'request_id': 'req-34b57179-1476-4f76-a324-712812b25386'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:39.332170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd56d6b673a4a74943ba094ae3f6cda', 'request_id': 'req-34b57179-1476-4f76-a324-712812b25386'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:39.355416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd56d6b673a4a74943ba094ae3f6cda', 'request_id': 'req-34b57179-1476-4f76-a324-712812b25386'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:39.376160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] The token cache is being invalidate because project cfd56d6b673a4a74943ba094ae3f6cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:39.376849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:39.379846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-34b57179-1476-4f76-a324-712812b25386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:39.389709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 837/1667] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:39 2022] DELETE /identity/v3/projects/cfd56d6b673a4a74943ba094ae3f6cda => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:39.391608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.392959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.393474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.452842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:39.456218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.462765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337e708-d080-4bea-bcb9-32f5b968d786 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.475162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:39.475349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.475778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.476079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.476913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:39.478901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df4b12-817c-405c-aada-bb5ea5125458 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:39.501371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 831/1668] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:39.508906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab026b-e0e4-440a-94c5-93d0d8db543a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.509191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab026b-e0e4-440a-94c5-93d0d8db543a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.509468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab026b-e0e4-440a-94c5-93d0d8db543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.561425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ceab026b-e0e4-440a-94c5-93d0d8db543a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:39.570210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ceab026b-e0e4-440a-94c5-93d0d8db543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.592165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 832/1669] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:39.593720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.594128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.594519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.641429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] MFA Rules not processed for user `51657a0e4865469999bdf8cd5102dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:39.643099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.650155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 838/1670] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jul 27 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:39.651696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:39.655017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.695862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a37e-e2ca-4a54-9e20-3db4af0ae465 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.711120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:39.711978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.712237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.712520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] PATH_INFO: `/v3/projects/13affe4fe7f84d8b9cd60e156993af4b/users/c3fc8cb3ffb447fcbeb2b015df14cb5d/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.746382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13affe4fe7f84d8b9cd60e156993af4b, user_id=c3fc8cb3ffb447fcbeb2b015df14cb5d, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:39.752158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d10016-bc6a-4c56-b835-f6076b1bbe09 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:39.784137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 839/1671] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:39 2022] PUT /identity/v3/projects/13affe4fe7f84d8b9cd60e156993af4b/users/c3fc8cb3ffb447fcbeb2b015df14cb5d/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:39.785541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:39.785880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:39.786195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:39.801506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 833/1672] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:39.803590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:39.837024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff6c13-6c85-49da-a9fb-971298c5ffac None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:39.847222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:39.866910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:39.870339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:39.908511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:40.009087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.009367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.009595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.009870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.018695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:40.051535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.053946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9f5cd-eccf-42ef-8864-ad4667280863 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.074623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 834/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:40.081177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 840/1674] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:40.098444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.098883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.099175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.099452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.122379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d529-4f37-4145-9bc3-42ae65ba5395 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.135553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.136775 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.138505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.140194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.141123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:40.144192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.146939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.176727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 841/1675] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:40.180238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.201155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908347ca-b63e-4e10-a899-589e81b53ac1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.202995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] MFA Rules not processed for user `c3fc8cb3ffb447fcbeb2b015df14cb5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:40.204251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:40.214674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.215997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.217820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.218164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.220562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.225318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.290492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 842/1676] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:40.293498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.418805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8245a-00b1-4272-ba61-330112c05343 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.424276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.425075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.425444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.425900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.426456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.429983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.434231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 835/1677] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:40.449002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.474463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd5e0-2709-44b1-9712-34f38a170e2a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.487886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 843/1678] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/users => generated 279 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:40.491933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.492236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.492576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.492855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.493128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.493798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.495720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.507761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc73737-9cc1-4e3f-a57e-093cd7172d7e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.530001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.531419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.532287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.533031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.534418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.536758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.554176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 844/1679] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:28:40.557955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.580174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 836/1680] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:40.581858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.588145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11138c5-3699-44b4-9afa-5288db0aa198 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.749821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:40.759177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45547c-331e-40e2-af7f-00de4372e62a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.761074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.762220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.762773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.763498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.764811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.765839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.766498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.766785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.767065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.768073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.771936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.786147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.786950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 845/1681] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:28:40.791420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.809109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53183c6-efbf-4fb5-85d5-006a49e007f5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.814710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:40.822843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.823657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.823970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.824250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] PATH_INFO: `/v3/projects/afa19fcb601f475f9c9478b5084701d5/users/d6ea79543341478389dd108c2240168d/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.844776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 837/1682] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:40.847751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:40.849179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa19fcb601f475f9c9478b5084701d5, user_id=d6ea79543341478389dd108c2240168d, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.854998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d9353-c9ab-417b-8ea8-eb36e5769dc5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:40.873728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b4359-b2c1-4090-a35f-1399237e3e6a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:40.877848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 846/1683] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:40 2022] PUT /identity/v3/projects/afa19fcb601f475f9c9478b5084701d5/users/d6ea79543341478389dd108c2240168d/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:40.890797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.891112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.891694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.968956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] MFA Rules not processed for user `d6ea79543341478389dd108c2240168d`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:40.970475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:40.973954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:40.976135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:40.976450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:40.976740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:40.978965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:40.986121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.025414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 838/1684] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:41.038908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.066996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927991db-bf4e-4cf8-81ce-4fb3c8dab6b0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:41.081058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:41.082335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.082627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.082927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] PATH_INFO: `/v3/projects/d3ae135f4aa04cfba9e1688084c479f3/users/8b45173bc6404005b0f0b332d7010bd5/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.130553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ae135f4aa04cfba9e1688084c479f3, user_id=8b45173bc6404005b0f0b332d7010bd5, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:41.138646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.163065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 847/1685] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:40 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:41.165300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.174896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 839/1686] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:41 2022] PUT /identity/v3/projects/d3ae135f4aa04cfba9e1688084c479f3/users/8b45173bc6404005b0f0b332d7010bd5/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:41.179529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.186165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36479c16-348e-4e9d-b267-c2af803c14c3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:41.341900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:41.343714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.344093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.345756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.356230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:41.364485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98f4d2-1bd2-496d-8cc2-d5c8f5d66443 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.393045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd47ee-66d6-490b-906e-a60a3daf7f54 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:41.411601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:41.413413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.413701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.416849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.419069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:41.424064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8eef1-a99c-4528-a3b2-328d00fb1d5c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.432477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 848/1687] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:41 2022] POST /identity/v3/projects => generated 357 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:28:41.433945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11443be-0133-470b-af56-460c1368f559 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.434236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11443be-0133-470b-af56-460c1368f559 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.434509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11443be-0133-470b-af56-460c1368f559 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.436207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 849/1688] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 15:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:28:41.436907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11443be-0133-470b-af56-460c1368f559 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.463427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 840/1689] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:41.466676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.466676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.466836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.573519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:41.575986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:41.611967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:41.665606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11443be-0133-470b-af56-460c1368f559 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:41.697729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:41.698388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.698672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.698951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.705042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:41.711311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.767930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 841/1690] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:41.769564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.788947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d2b23-5192-466b-ad19-6515fe0bac79 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:41.797010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 850/1691] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:41 2022] POST /identity/v3/users => generated 285 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:41.798337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:41.798337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:41.798337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:41.798687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.798889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] PATH_INFO: `/v3/projects/d3ae135f4aa04cfba9e1688084c479f3/users/8b45173bc6404005b0f0b332d7010bd5/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:41.802516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ae135f4aa04cfba9e1688084c479f3, user_id=8b45173bc6404005b0f0b332d7010bd5, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:41.808466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:41.818507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:41.836284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 842/1692] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:41 2022] PUT /identity/v3/projects/d3ae135f4aa04cfba9e1688084c479f3/users/8b45173bc6404005b0f0b332d7010bd5/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:41.839499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:41.847162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:41.847643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb65e40-a383-42ca-8a56-31aa1e61cc52 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:42.174742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:42.184760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.185047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.185329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.186064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:42.187716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359334e-bcde-4f88-a1d2-9d553914d5c2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:42.229592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f75745-c647-4d97-936b-c82cf13b8ac8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:42.244312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 851/1693] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:42.245647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.245944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.246220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.256467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:42.259513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.262120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.262434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.292652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:42.325528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:42.364556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] MFA Rules not processed for user `8b45173bc6404005b0f0b332d7010bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:42.367271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 843/1694] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:42.368217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:42.376688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:42.405901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa182202-954a-4d97-861b-9dca2b5e682d service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:42.433420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:42.437845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.440227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.440935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] PATH_INFO: `/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e/users/fea34bb8087d4db692174551c9273f99/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.501336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6715f22d6c5c4afb825c2b9b0be9ac1e, user_id=fea34bb8087d4db692174551c9273f99, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:42.504348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:42.553732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 844/1695] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:42 2022] PUT /identity/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e/users/fea34bb8087d4db692174551c9273f99/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:42.555223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:42.560048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:42.625407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 852/1696] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:42.626293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:42.680882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc06f9f-4495-470a-a91d-cf8f6a9d254a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:42.799659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dae17-803b-42fd-8d98-f84c9c3c0275 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:42.813605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:42.814504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.814980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.815451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] PATH_INFO: `/v3/users/571667712bee4f87a22fff9490151b5a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.820801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571667712bee4f87a22fff9490151b5a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:42.822073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 15:28:42.822073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 27 15:28:42.822073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: Jul 27 15:28:42.826084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:42.856141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:42.874163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:42.875993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:42.876252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:42.876534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:42.877320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:42.879617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:42.898513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 853/1697] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:42.899441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:42.901222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:42.999355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571667712bee4f87a22fff9490151b5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:43.030557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571667712bee4f87a22fff9490151b5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:43.042150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b1a6f-2b8f-4a5b-8bc0-1bf816b3e45d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:43.045980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:43.046613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.046895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.047167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.110560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571667712bee4f87a22fff9490151b5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:43.145960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:43.178473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571667712bee4f87a22fff9490151b5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:43.181516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eb7c82b-50dc-4d7a-ad67-85776c223634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571667712bee4f87a22fff9490151b5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:43.185401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:43.187453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:43.202958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 854/1698] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:43.203643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:43.226192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25e4e2-a8d7-4184-9079-25cfe84a9cd6 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:43.234315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 845/1699] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:42 2022] DELETE /identity/v3/users/571667712bee4f87a22fff9490151b5a => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:43.247765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.248108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.248655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.316352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:43.317049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.317339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.317821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] PATH_INFO: `/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e/users/fea34bb8087d4db692174551c9273f99/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.321855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6715f22d6c5c4afb825c2b9b0be9ac1e, user_id=fea34bb8087d4db692174551c9273f99, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:43.324645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056686d-d398-4d40-9580-74bf601f44be admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:43.340887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 855/1700] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:43 2022] PUT /identity/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e/users/fea34bb8087d4db692174551c9273f99/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:43.342899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5867-8633-4ee5-a835-0314d8a77f8f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.343479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5867-8633-4ee5-a835-0314d8a77f8f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.343767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5867-8633-4ee5-a835-0314d8a77f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.375639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-71fb5867-8633-4ee5-a835-0314d8a77f8f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:43.376956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-71fb5867-8633-4ee5-a835-0314d8a77f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:43.394007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] MFA Rules not processed for user `a2b027a2c8dd4459b1309faab28f3d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:43.396777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:43.438217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:43.543778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 856/1701] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:43.545305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baad73d-53e3-462a-b6c4-946b93271935 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.545592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baad73d-53e3-462a-b6c4-946b93271935 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.545872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baad73d-53e3-462a-b6c4-946b93271935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.690772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8baad73d-53e3-462a-b6c4-946b93271935 None None] MFA Rules not processed for user `fea34bb8087d4db692174551c9273f99`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:43.692047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8baad73d-53e3-462a-b6c4-946b93271935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:43.707478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 846/1702] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:43.707478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:43.707478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:43.877513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0594b-c7c9-454e-9d99-aa7cd770dfc4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:43.899490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:43.900312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:43.900541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:43.900849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:43.934009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:43.935108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:43.972839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 847/1703] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:43.979692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:43.986030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.018308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 857/1704] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:44.020056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:44.020348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:44.020627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:44.160968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:44.164061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.332053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ffe5c-bc95-4d62-9636-f6f4a5dca7ba service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:44.349306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:44.350201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:44.350628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:44.352511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:44.377370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 858/1705] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:28:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:44.379570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:44.384851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:44.385736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:44.386268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.415077 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 848/1706] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:44.420190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:44.425964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.491301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e16fa-0e1f-4c09-8680-d0d9734160c5 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:44.506692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:44.507356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:44.507638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:44.507951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] PATH_INFO: `/v3/users/c3fc8cb3ffb447fcbeb2b015df14cb5d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:44.509281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3fc8cb3ffb447fcbeb2b015df14cb5d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:44.513807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:44.535691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:44.556567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:44.634418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c8bc9-a140-4a25-9911-2087435e7410 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:44.650310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:44.650958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:44.651244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:44.651515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:44.664200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3fc8cb3ffb447fcbeb2b015df14cb5d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.680686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:44.682329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:44.686164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3fc8cb3ffb447fcbeb2b015df14cb5d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.715556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 859/1707] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:44.717650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:44.723793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.731555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3fc8cb3ffb447fcbeb2b015df14cb5d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.755733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3fc8cb3ffb447fcbeb2b015df14cb5d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.763130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca433-9129-48d1-a8b8-07b7473410e3 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:44.763644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3fc8cb3ffb447fcbeb2b015df14cb5d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.778197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:44.778472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 849/1708] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:44 2022] DELETE /identity/v3/users/c3fc8cb3ffb447fcbeb2b015df14cb5d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:44.779279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:44.779742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:44.780219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] PATH_INFO: `/v3/users/4593106ecf604fe891b7a17c21141c24` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:44.783376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4593106ecf604fe891b7a17c21141c24)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:44.790212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:44.792985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:44.793549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:44.902462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4593106ecf604fe891b7a17c21141c24'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.903940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4593106ecf604fe891b7a17c21141c24'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.916425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4593106ecf604fe891b7a17c21141c24'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.936763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4593106ecf604fe891b7a17c21141c24'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:44.982108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-06dd3bd8-3d38-46cd-b153-2243b10e44b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4593106ecf604fe891b7a17c21141c24'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:45.001657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 860/1709] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:44 2022] DELETE /identity/v3/users/4593106ecf604fe891b7a17c21141c24 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:45.001657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:45.001657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:45.001657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:45.019008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cdbb1-a3d2-46f7-bcc4-d904a39bb6a4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:45.056536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:45.059228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.170416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:45.172432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:45.173122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:45.173404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:45.178552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.194443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 861/1710] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:45.195484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:45.199416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.236254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:45.378898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:45.384729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:45.389487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca002d-a68b-4dee-9256-3549e88cb504 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:45.390338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd3bf-ef42-42a2-baa3-0da13344a18d service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:45.401106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:45.402134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:45.403908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:45.404723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:45.412581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.416417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 850/1711] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:28:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:28:45.418465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f79d78-c37a-40a2-bfc0-0d84f2e29b1e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:45.419534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f79d78-c37a-40a2-bfc0-0d84f2e29b1e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:45.420924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f79d78-c37a-40a2-bfc0-0d84f2e29b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:45.484695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c4f79d78-c37a-40a2-bfc0-0d84f2e29b1e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:45.488788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c4f79d78-c37a-40a2-bfc0-0d84f2e29b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.508511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 851/1712] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:45.514437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:45.515148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:45.516876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:45.634005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] MFA Rules not processed for user `1989ddc458d645af8d017987a6b93809`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:45.637729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:45.732410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:45.736607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:45.778988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 862/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:45.781855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:45.785227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:46.009586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3df0-5558-471a-8e96-85ee6cf00f8a service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:46.019375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:46.022982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:46.023260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:46.023534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:46.101919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 852/1714] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 593 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:46.102873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:46.112563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:46.118053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4095e6-ea5c-486d-837e-84e5081d4e14 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:46.149996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed36363-77db-4c77-b242-22e4291dac0c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:46.170579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 863/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:46.174202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:46.176435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:46.177132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:46.178269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:46.178380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:46.178480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:46.178766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:46.263185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:46.264672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:46.292858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] MFA Rules not processed for user `1989ddc458d645af8d017987a6b93809`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:46.305103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:46.341193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 864/1716] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:46.341386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 853/1717] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:46.353797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:46.355889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:46.582234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c907ac-f615-4faf-830f-e0e83bb08179 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:46.618628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:46.620124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:46.620448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:46.620804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:46.643581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:46.645149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:46.690251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 865/1718] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:48.785678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:48.791517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:48.847384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e82d2-43b9-48ca-a51b-790045576c9d service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:48.875259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:48.883004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:48.884977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:48.885870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] PATH_INFO: `/v3/projects/ee338fc85c214e2da3127c44b710cefe` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:48.892101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee338fc85c214e2da3127c44b710cefe)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:48.897256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:49.122838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee338fc85c214e2da3127c44b710cefe', 'request_id': 'req-5dc098b5-448a-4cc4-92bf-6d7291debf7d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.152087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee338fc85c214e2da3127c44b710cefe', 'request_id': 'req-5dc098b5-448a-4cc4-92bf-6d7291debf7d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.181964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee338fc85c214e2da3127c44b710cefe', 'request_id': 'req-5dc098b5-448a-4cc4-92bf-6d7291debf7d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.225794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] The token cache is being invalidate because project ee338fc85c214e2da3127c44b710cefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:49.232212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.235593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5dc098b5-448a-4cc4-92bf-6d7291debf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.240238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 854/1719] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:48 2022] DELETE /identity/v3/projects/ee338fc85c214e2da3127c44b710cefe => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:49.246077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:49.283806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcf3c2-a3f9-437e-a931-2a66ae2de9fa service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:49.301313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065594-da10-4235-8208-d39fc09e0895 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:49.301636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065594-da10-4235-8208-d39fc09e0895 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:49.301868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065594-da10-4235-8208-d39fc09e0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:49.312533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:49.423577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ff065594-da10-4235-8208-d39fc09e0895 None None] MFA Rules not processed for user `7a964a003f8e4f1da603a13eb7f534ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:49.430336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ff065594-da10-4235-8208-d39fc09e0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:49.447566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:49.681365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:49.692696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:49.696614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:49.696614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] PATH_INFO: `/v3/users/ce9f4146e0ca471a882ccf9b99c75a57` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:49.698013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9f4146e0ca471a882ccf9b99c75a57)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:49.705439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 855/1720] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:49.705946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:49.714386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:49.715626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:49.717952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:49.791539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:49.801980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:49.900012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9f4146e0ca471a882ccf9b99c75a57'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.910959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9f4146e0ca471a882ccf9b99c75a57'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.951629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9f4146e0ca471a882ccf9b99c75a57'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:49.973545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9f4146e0ca471a882ccf9b99c75a57'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.032239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 856/1721] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:50.033511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:50.049559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9f4146e0ca471a882ccf9b99c75a57'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.062619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.079795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 866/1722] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:49 2022] DELETE /identity/v3/users/ce9f4146e0ca471a882ccf9b99c75a57 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:50.081783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:50.094275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.097258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b7a5c-e2f2-424d-9607-13eb7e41ba3b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:50.235195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:50.236220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:50.236791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:50.237278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] PATH_INFO: `/v3/projects/85b0f788133b43dba1ddcb9be9d9c6f7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:50.239012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b0f788133b43dba1ddcb9be9d9c6f7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:50.242172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:50.361330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:50.372210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b0f788133b43dba1ddcb9be9d9c6f7', 'request_id': 'req-d503c387-7d4a-419e-bfaa-89a8d1676481'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.390008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b0f788133b43dba1ddcb9be9d9c6f7', 'request_id': 'req-d503c387-7d4a-419e-bfaa-89a8d1676481'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.409161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfdd8d-813d-4422-833a-8e61daa33a6d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:50.413325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b0f788133b43dba1ddcb9be9d9c6f7', 'request_id': 'req-d503c387-7d4a-419e-bfaa-89a8d1676481'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.444865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] The token cache is being invalidate because project 85b0f788133b43dba1ddcb9be9d9c6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:50.446043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.447528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:50.456772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 857/1723] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:50 2022] DELETE /identity/v3/projects/85b0f788133b43dba1ddcb9be9d9c6f7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:50.458478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:50.464716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.557641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:50.700281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503c387-7d4a-419e-bfaa-89a8d1676481 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:50.708304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:50.709641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:50.710062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:50.710817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:50.714186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.719636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:50.720352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:50.720637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:50.720915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:50.726961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.764401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:50.766058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdb640-4ebd-4f07-93e3-59657d8cf83c service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:50.785612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 858/1724] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:50.789197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012a92-5e59-4565-91e1-8501b5348b46 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:50.789480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012a92-5e59-4565-91e1-8501b5348b46 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:50.789762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012a92-5e59-4565-91e1-8501b5348b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:50.877471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-24012a92-5e59-4565-91e1-8501b5348b46 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:50.883763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-24012a92-5e59-4565-91e1-8501b5348b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:50.947371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:50.949633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:50.972456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 867/1725] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:50.975368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:50.978221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:51.204595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 859/1726] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:51.211346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:51.211643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:51.212001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:51.269995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa328b-3b7b-46d8-8249-3f3e2b1ebb86 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:51.293758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:51.296263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:51.298571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:51.301919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] PATH_INFO: `/v3/projects/13affe4fe7f84d8b9cd60e156993af4b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:51.306696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:51.310148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:51.311908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13affe4fe7f84d8b9cd60e156993af4b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:51.327998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:51.364599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 860/1727] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:51.373087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:51.405534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c420ea-786c-4cc8-963a-19b4b8abe490 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:51.467955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13affe4fe7f84d8b9cd60e156993af4b', 'request_id': 'req-44030a0e-32b4-48b4-9696-90bc1bcc1400'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:51.483780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13affe4fe7f84d8b9cd60e156993af4b', 'request_id': 'req-44030a0e-32b4-48b4-9696-90bc1bcc1400'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:51.506982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13affe4fe7f84d8b9cd60e156993af4b', 'request_id': 'req-44030a0e-32b4-48b4-9696-90bc1bcc1400'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:51.572145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] The token cache is being invalidate because project 13affe4fe7f84d8b9cd60e156993af4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:51.572690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:51.573514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:51.575938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 868/1728] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:50 2022] DELETE /identity/v3/projects/13affe4fe7f84d8b9cd60e156993af4b => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:51.580318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:51.582711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:51.599703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:51.634223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:51.635016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:51.635512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:51.635856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:51.636837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:51.637305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:51.639262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:51.694483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 861/1729] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:51.715247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:51.726307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:51.749949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:51.772505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b411ca-4b16-493f-a3d1-9a18b54589d7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:51.790945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44030a0e-32b4-48b4-9696-90bc1bcc1400 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:51.791166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:51.804141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:51.804783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:51.805126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:51.807551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:51.812317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:51.823521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:51.824316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:51.824655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:51.825005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] PATH_INFO: `/v3/users/8b45173bc6404005b0f0b332d7010bd5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:51.826409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b45173bc6404005b0f0b332d7010bd5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:51.835665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:51.924927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 862/1730] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:51 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:51.929991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:52.056434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b45173bc6404005b0f0b332d7010bd5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:52.060709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b45173bc6404005b0f0b332d7010bd5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:52.089526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b45173bc6404005b0f0b332d7010bd5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:52.112339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b45173bc6404005b0f0b332d7010bd5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:52.174395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b45173bc6404005b0f0b332d7010bd5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:52.206773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 869/1731] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:51 2022] DELETE /identity/v3/users/8b45173bc6404005b0f0b332d7010bd5 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:28:52.264790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1e566-2325-42a8-b2f7-b64e34f75c87 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:52.463750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:52.466220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:52.466983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:52.467278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:52.468144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:52.473428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:52.515878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 863/1732] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:51 2022] POST /identity/v3/users => generated 288 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:28:52.517453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:52.519099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:52.580442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b077-85a8-4487-b5d7-50cd3aa08d29 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:52.628693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:52.629355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:52.629643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:52.629922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:52.654812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:52.665991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:52.747870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 870/1733] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:28:52.751815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:52.765310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfd299-c374-4dc3-a080-70f3ccfae346 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:52.791495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:52.791803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:52.795352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:52.795787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:52.796644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:52.798439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1e357-e6c2-4519-a935-6a7964fac890 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:52.870203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 864/1734] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:52.871610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:52.886402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb955094-9ff7-4484-aac8-4cf5cae1be05 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:52.935055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:52.935767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:52.936471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:52.936600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] PATH_INFO: `/v3/projects/d87aff68ffd043c28325263d420856dd/users/67617ddbde374dca87405daae7952920/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:53.010471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87aff68ffd043c28325263d420856dd, user_id=67617ddbde374dca87405daae7952920, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:53.018015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:53.079571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 871/1735] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:52 2022] PUT /identity/v3/projects/d87aff68ffd043c28325263d420856dd/users/67617ddbde374dca87405daae7952920/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:28:53.090916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:53.091719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:53.091719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:53.157747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] MFA Rules not processed for user `67617ddbde374dca87405daae7952920`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:53.159207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:53.310002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:53.335459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 865/1736] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:53.354493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:53.356466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:53.536403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:53.610526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439ea25-ef96-40c9-a8b4-8d014d8bbfc6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:53.636825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:53.637845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:53.638417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:53.638891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:53.663764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:53.669357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271295d-8d88-434d-a15e-5cfe1ed1fd31 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:53.698143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 872/1737] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:28:55.340472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:55.550385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03f7f3-79c5-4470-bf16-c9c4a274dcac None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:55.558946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:55.559625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:55.560137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:55.560560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] PATH_INFO: `/v3/users/67617ddbde374dca87405daae7952920` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:55.565751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67617ddbde374dca87405daae7952920)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:55.569407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:55.709244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67617ddbde374dca87405daae7952920'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:55.725845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67617ddbde374dca87405daae7952920'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:55.765754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67617ddbde374dca87405daae7952920'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:55.787398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67617ddbde374dca87405daae7952920'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:55.788185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67617ddbde374dca87405daae7952920'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:55.812412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 866/1738] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:28:55 2022] DELETE /identity/v3/users/67617ddbde374dca87405daae7952920 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:55.819721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0450-4e3c-4a17-81f6-a436dbd9a9a6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:55.820319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0450-4e3c-4a17-81f6-a436dbd9a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:55.820719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0450-4e3c-4a17-81f6-a436dbd9a9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:55.888585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:55.893867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:55.914330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-384c0450-4e3c-4a17-81f6-a436dbd9a9a6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:55.918330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-384c0450-4e3c-4a17-81f6-a436dbd9a9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.039028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 873/1739] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:56.042693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.043328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.043853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.111063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] MFA Rules not processed for user `a2b027a2c8dd4459b1309faab28f3d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:56.113790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.136130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ba4ed-4b25-4551-9cb7-bb0dc399eb82 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:56.149014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:56.149769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.150153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.150446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] PATH_INFO: `/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.152725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c33bfd34b2cb49b4ae969afb9abb68ea)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:56.160926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:56.201574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:56.298065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c33bfd34b2cb49b4ae969afb9abb68ea', 'request_id': 'req-7594a073-ac3f-48ff-bdae-28e0067825b7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:56.334982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 874/1740] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:56.350771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c33bfd34b2cb49b4ae969afb9abb68ea', 'request_id': 'req-7594a073-ac3f-48ff-bdae-28e0067825b7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:56.366238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:56.368488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.389922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c33bfd34b2cb49b4ae969afb9abb68ea', 'request_id': 'req-7594a073-ac3f-48ff-bdae-28e0067825b7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:56.450832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] The token cache is being invalidate because project c33bfd34b2cb49b4ae969afb9abb68ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:56.451128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:56.458008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7594a073-ac3f-48ff-bdae-28e0067825b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:56.469367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 867/1741] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:55 2022] DELETE /identity/v3/projects/c33bfd34b2cb49b4ae969afb9abb68ea => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:56.474898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f7cb-dd4c-46c3-b9fe-491daa269623 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.483584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f7cb-dd4c-46c3-b9fe-491daa269623 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.484156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f7cb-dd4c-46c3-b9fe-491daa269623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.550655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a563f7cb-dd4c-46c3-b9fe-491daa269623 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:56.552165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a563f7cb-dd4c-46c3-b9fe-491daa269623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.678002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f913a5e-dee9-4225-8d8e-31427c13b444 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:56.754523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 868/1742] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:56.764429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.766144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.766566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.832354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:56.834840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.843030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:56.846039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 869/1743] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:28:56.851925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:56.869022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:56.871638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.874598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.875152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.878889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:56.893797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863b830-5691-4c09-b42c-8812b13a7b00 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:56.909413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:56.911543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:56.917325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:56.918122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:56.932630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:56.937484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:56.938268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:28:56.943192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:56.945237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 875/1744] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:28:56.946876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.034998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 870/1745] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:28:57.039653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:57.052862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72f3b1-8bd2-4cb5-a38e-dc32b65522a1 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:57.066424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:57.067203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.067494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.068406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.069284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:57.075480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.141213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 876/1746] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:57 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:57.144898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:57.169365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6e3d-3b40-4a74-aa63-1a602481651e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:57.336912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:57.337775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.338328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.338900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.340196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:57.346457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.401283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 871/1747] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:57 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:57.405198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:57.423072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbaf1a-510c-47e3-94fd-b25f59fecb88 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:57.434578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:57.437460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.438588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.439391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.441548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:57.447334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.486841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 877/1748] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:57.491475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:57.509123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045994a-94cd-4383-b79e-f0ad305385d3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:57.526375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:57.533222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.533918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.535004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.537227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:57.543726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a21685-851c-4d93-84e5-b6129a779736 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.572889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 872/1749] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:57.575256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:57.591787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c756c-a15c-4145-a4af-cd806166cc4b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:57.627349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:57.628349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.633762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.634039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] PATH_INFO: `/v3/projects/d2a3c1d60555442d82dcf062290a6155/users/1a9042133fda41768c32d3bf62e65ae3/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.674756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a3c1d60555442d82dcf062290a6155, user_id=1a9042133fda41768c32d3bf62e65ae3, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:57.681243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:57.736297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 878/1750] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:57 2022] PUT /identity/v3/projects/d2a3c1d60555442d82dcf062290a6155/users/1a9042133fda41768c32d3bf62e65ae3/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:28:57.738041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd4dd-4597-465e-9df3-9fc828175885 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:57.738337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd4dd-4597-465e-9df3-9fc828175885 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:57.738619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd4dd-4597-465e-9df3-9fc828175885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:57.754783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:57.789042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6fbdd4dd-4597-465e-9df3-9fc828175885 None None] MFA Rules not processed for user `1a9042133fda41768c32d3bf62e65ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:57.791196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6fbdd4dd-4597-465e-9df3-9fc828175885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:58.049346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:58.051151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 873/1751] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:58.077165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993cfc7-4efe-44a9-8fec-995a1abd0749 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.077480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993cfc7-4efe-44a9-8fec-995a1abd0749 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.079694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993cfc7-4efe-44a9-8fec-995a1abd0749 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.082966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 874/1752] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:28:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:28:58.115437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.115721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.116032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.192481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:58.197658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:58.217578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910432-45da-407b-854e-18cf3fe9da8a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:58.232731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:58.233386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.233672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.233949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.234635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:58.236603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:58.371236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 879/1753] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:28:58 2022] POST /identity/v3/projects => generated 375 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:58.375164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:58.470862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 875/1754] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:58.553945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357aaad-6437-490e-9c51-e8bc53671775 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:58.556752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:58.558394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:58.575848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:58.577758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.577877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.578128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.578851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:58.581966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b497e-feb2-481d-90c9-4477b25f9580 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:58.589405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:58.643158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:58.657951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 880/1755] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:28:58 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:28:58.661014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:58.703490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae211e-c123-4003-8bee-40e8d821570f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:58.722685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:58.723341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.726033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.726389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.727230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:58.742993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:58.783055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 881/1756] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:58.788342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:58.814192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559423e-d056-42d6-b7da-0f5981a4ef61 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:58.829632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:58.830928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.831614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.833070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.839041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:58.839744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:58.840109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:58.840392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] PATH_INFO: `/v3/projects/6243acf945244fd184006f2a0f07fefa/users/861085a695924e578becfa35feeae78d/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:58.853513 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:58.857896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:58.899797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6243acf945244fd184006f2a0f07fefa, user_id=861085a695924e578becfa35feeae78d, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:58.902901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 876/1757] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:28:58.903960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03221f83-5c89-4002-90fa-f9bf7716f24d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:58.940888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 882/1758] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:58 2022] PUT /identity/v3/projects/6243acf945244fd184006f2a0f07fefa/users/861085a695924e578becfa35feeae78d/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:28:58.943570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:59.010475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcf61c-cc33-4629-ad1b-0d21738e4d97 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:59.083756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:59.331977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:59.333949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.338762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.341626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.344520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:59.355889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:59.388120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.388700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.389148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.391231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 883/1759] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:28:59.413706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:59.417080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:59.444890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 877/1760] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:28:59.448573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:59.476659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcacd1-8fdc-42dc-9a7f-68176997d326 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:59.487467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af78ae-25f0-436d-8ff7-9d85f63a5302 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:59.502276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:59.502978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.503263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.503545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] PATH_INFO: `/v3/projects/d87aff68ffd043c28325263d420856dd` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.505048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87aff68ffd043c28325263d420856dd)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:59.507746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:59.513003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:59.513750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.514036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.514339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] PATH_INFO: `/v3/projects/6243acf945244fd184006f2a0f07fefa/users/861085a695924e578becfa35feeae78d/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.517307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6243acf945244fd184006f2a0f07fefa, user_id=861085a695924e578becfa35feeae78d, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:59.520337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f3d45-50d6-4a79-a9db-84023b23090a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:28:59.557694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 878/1761] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:28:59 2022] PUT /identity/v3/projects/6243acf945244fd184006f2a0f07fefa/users/861085a695924e578becfa35feeae78d/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:59.574739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83f30-ebca-4877-b96d-368357019e80 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.575308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83f30-ebca-4877-b96d-368357019e80 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.576166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83f30-ebca-4877-b96d-368357019e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.639727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-45a83f30-ebca-4877-b96d-368357019e80 None None] MFA Rules not processed for user `861085a695924e578becfa35feeae78d`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:59.641313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-45a83f30-ebca-4877-b96d-368357019e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:59.641781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87aff68ffd043c28325263d420856dd', 'request_id': 'req-057961bf-15a6-4629-9c5a-bd1ef780ecaf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:59.663549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87aff68ffd043c28325263d420856dd', 'request_id': 'req-057961bf-15a6-4629-9c5a-bd1ef780ecaf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:59.694984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87aff68ffd043c28325263d420856dd', 'request_id': 'req-057961bf-15a6-4629-9c5a-bd1ef780ecaf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:59.717592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] The token cache is being invalidate because project d87aff68ffd043c28325263d420856dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:28:59.718271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:59.719197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-057961bf-15a6-4629-9c5a-bd1ef780ecaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:28:59.725731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 884/1762] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:59 2022] DELETE /identity/v3/projects/d87aff68ffd043c28325263d420856dd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:28:59.727678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.728139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.728676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.754975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:59.756834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:59.841935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 879/1763] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:59.845641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.845641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.845641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.879074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 885/1764] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:28:59.880027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:28:59.882029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:59.892907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:28:59.897153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:28:59.946558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:28:59.988452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94999d-7355-4935-9f34-2a44749c5444 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:28:59.994449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:28:59.995115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:28:59.995403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:28:59.995679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] PATH_INFO: `/v3/projects/d3ae135f4aa04cfba9e1688084c479f3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:28:59.996948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ae135f4aa04cfba9e1688084c479f3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:28:59.999916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.044218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 880/1765] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:00.047009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.049647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:00.090067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:00.105050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ae135f4aa04cfba9e1688084c479f3', 'request_id': 'req-2138556a-3ba4-46a4-9f05-6b8829fae3c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:00.125867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ae135f4aa04cfba9e1688084c479f3', 'request_id': 'req-2138556a-3ba4-46a4-9f05-6b8829fae3c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:00.140482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f1b34-926b-4749-ad96-442f787c3a34 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.141418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ae135f4aa04cfba9e1688084c479f3', 'request_id': 'req-2138556a-3ba4-46a4-9f05-6b8829fae3c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:00.165648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] The token cache is being invalidate because project d3ae135f4aa04cfba9e1688084c479f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:00.166303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:00.168181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2138556a-3ba4-46a4-9f05-6b8829fae3c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:00.179049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 886/1766] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:28:59 2022] DELETE /identity/v3/projects/d3ae135f4aa04cfba9e1688084c479f3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:00.180578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.183474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.183772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.208007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.209802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.210399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.210892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.213014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:00.230858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:00.232038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:00.291722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.293678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a25314-5f60-46f4-a23c-87b0b7cfeb0d service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.314142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 881/1767] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:00.315778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7fa20-cb17-4c24-bd71-80ff9bc43655 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.316208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7fa20-cb17-4c24-bd71-80ff9bc43655 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.316741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7fa20-cb17-4c24-bd71-80ff9bc43655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.385622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 887/1768] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:00.386632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.397394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-86d7fa20-cb17-4c24-bd71-80ff9bc43655 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:00.397793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6bb8-60b6-48f3-b9ba-6ee2c9fac314 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.401120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86d7fa20-cb17-4c24-bd71-80ff9bc43655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:00.403572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.404368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.404770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.405155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.405994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:00.406450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.409065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.423719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 882/1769] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:00.433219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.433341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.433620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.456023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 888/1770] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:00.457343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.479156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508883-0496-4423-8da7-644d1c649101 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.485606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:00.488108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:00.490774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.492852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.493720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.495373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.497524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.504521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.518086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 883/1771] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:00.523423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.557593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63b79-06b5-4ff7-bcab-57e993ec992c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.571098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.571857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.574203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.574487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.575230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:00.578567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 889/1772] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:00.579581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.579712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.584230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.629966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 884/1773] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:00.650768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.672692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce25d-5f27-48c9-9aff-2bcb8a00b6ee admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.682786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:00.751923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7deb73-42d7-41ed-bcd0-8c427bae3c35 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.765717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.766635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.766784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.767110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.767879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.781891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.835910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 890/1774] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:00.842739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.854344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.855624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.857331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.857636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.859566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.865510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7d84f-7108-4880-8ab8-f7bf4afaa60d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:00.869032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.887463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:00.888599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:00.888998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:00.892182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:00.893059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:00.894941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:00.899652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:00.930474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 891/1775] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:00.934204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.956646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 885/1776] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/projects => generated 393 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:00.961320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:00.967367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0a0c-0b45-4fbf-8e1b-71209f18fbe3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.197892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b704-4a91-4cb3-a738-472770f07474 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.216563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.217373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.217645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.217926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.219309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.221495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.231908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.232807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.233196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.234332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.240240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.243262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.281107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 892/1777] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:01.296671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.315963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 886/1778] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:00 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:01.322370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.332169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177b30a-cc80-48f2-ba10-ea880c54b610 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.356829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6f2e0-40fc-4463-9842-fd1b8e20d995 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.362588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.363440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.363931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.364286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] PATH_INFO: `/v3/projects/de92eeb0d263464ea91f5d04f7f8c0d0/users/a106467c95da4254b15875c3c639552f/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.372377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.373275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.373710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.374127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.375150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.378961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.424788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de92eeb0d263464ea91f5d04f7f8c0d0, user_id=a106467c95da4254b15875c3c639552f, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.425213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 887/1779] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:01.427197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.434083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd7bca-bda6-499f-a118-1ae8134e5be3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.471284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82423b6d-56fa-4fe3-a1ea-201669b0fbe6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.487202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.488557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.489096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.489096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.499336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 893/1780] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:01 2022] PUT /identity/v3/projects/de92eeb0d263464ea91f5d04f7f8c0d0/users/a106467c95da4254b15875c3c639552f/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:01.499439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.499439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.503032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.504061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.504424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.532428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 888/1781] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:01.533669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.560591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0dcb-a1ee-4039-b0b3-9b62d25fcd08 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.565342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] MFA Rules not processed for user `a106467c95da4254b15875c3c639552f`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:01.566971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:01.728628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.729450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.729736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.730474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] PATH_INFO: `/v3/projects/8c8e0d1e0dae4ce2949c98021480e5fa/users/44d53ede50424e0492def4d530c7ca76/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.759334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 894/1782] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:01.763714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.769702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8e0d1e0dae4ce2949c98021480e5fa, user_id=44d53ede50424e0492def4d530c7ca76, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.779395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.787210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc99ced-eb6e-48c6-b842-965ebd1bff95 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:01.799702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:01.800467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.801456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.801821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.802738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:01.804939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e99ddf-b4fd-4e6a-9aee-5233757fb8b8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:01.831064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 889/1783] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:01 2022] PUT /identity/v3/projects/8c8e0d1e0dae4ce2949c98021480e5fa/users/44d53ede50424e0492def4d530c7ca76/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:01.832307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:01.836041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:01.840902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:01.886786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 895/1784] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:01 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:01.888856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:01.888856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:01.889073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:01.954407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:01.960728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] MFA Rules not processed for user `44d53ede50424e0492def4d530c7ca76`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:01.962628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:02.043634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610d1be-2e81-4c15-b440-99b347def769 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.109903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 896/1785] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:02.111764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.264423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.265869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.266148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.266536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.276923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:02.319606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954f97-ec61-4e24-bc67-fc9c9cdfcd13 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.329029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.329679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.331127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.333670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.333962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.334243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.334934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.337335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d85c7e-7120-4a51-be7b-a2a75481d04a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.365766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 890/1786] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:02.367908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.390749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 897/1787] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:02 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:02.396099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.396924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.398402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.409643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3f0ea-a927-4171-90b7-8e12c23ff115 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.430637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.436258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.436850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.438349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.439080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.449047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.463006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] MFA Rules not processed for user `7a964a003f8e4f1da603a13eb7f534ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:02.471480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:02.516804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 891/1788] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:02 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:02.517809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.633587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce89a8-039b-45dd-b916-bfcfde0f954a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.657510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.658318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.658806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.658988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.660101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.663128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.670679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 898/1789] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:02.672757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.695606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 892/1790] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:02.695815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbe592-b33e-4455-9849-3a23176e36e5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.699709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.704771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.705414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.705696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.705971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.706648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.709397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.759671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:02.763623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25678a-e203-4d13-9ae0-f58c6ceab705 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.766235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 899/1791] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:02 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:02.769944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.792661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.795335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.797865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.801673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] PATH_INFO: `/v3/projects/774007a9c4384a0ab719ed271e9873d5/users/a980255e293e415b997dac71fb8e99f0/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.806673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac0ab3-1e65-4ad0-8ba6-dcff235df9f9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.815600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.816911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.817169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.817811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.818638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.820872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.837184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 900/1792] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:02.842399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.859146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38c58-58b5-4703-acdd-ba03b7e6f3ad admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:02.865225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:02.866450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:02.866960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:02.867936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] PATH_INFO: `/v3/projects/fd695a014b6c4f07845cc455545feb2d/users/cf11538858014581989eec80d13b9479/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:02.906207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774007a9c4384a0ab719ed271e9873d5, user_id=a980255e293e415b997dac71fb8e99f0, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.912151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.913943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd695a014b6c4f07845cc455545feb2d, user_id=cf11538858014581989eec80d13b9479, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:02.921114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:02.954376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 893/1793] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:02 2022] PUT /identity/v3/projects/774007a9c4384a0ab719ed271e9873d5/users/a980255e293e415b997dac71fb8e99f0/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:02.959683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:02.976830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 901/1794] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:02 2022] PUT /identity/v3/projects/fd695a014b6c4f07845cc455545feb2d/users/cf11538858014581989eec80d13b9479/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:02.978305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.009828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:03.218932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1a5e-6a07-458b-88cc-b7758c2c40d6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.227173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.227947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.228179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e179-6047-4690-9877-0c2a1fca616c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.228356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.229316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.230109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.236902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.241064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.241717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.242001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.242280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.242992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.254220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.259386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 894/1795] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:03.262662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.274581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 902/1796] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:03.283577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.296467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cfc1c-cfd7-4682-850b-9267bfe6f750 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.309005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a384b8e-4bc2-4eb5-8e6e-a30a445e6a4c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.314021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.314723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.315007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.315289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] PATH_INFO: `/v3/projects/fd695a014b6c4f07845cc455545feb2d/users/cf11538858014581989eec80d13b9479/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.320000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd695a014b6c4f07845cc455545feb2d, user_id=cf11538858014581989eec80d13b9479, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.322076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.323353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779d00e-943e-41d3-821e-4c303b5fd312 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.323485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.323485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.324461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] PATH_INFO: `/v3/projects/774007a9c4384a0ab719ed271e9873d5/users/a980255e293e415b997dac71fb8e99f0/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.334344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774007a9c4384a0ab719ed271e9873d5, user_id=a980255e293e415b997dac71fb8e99f0, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.337896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ee07e-b7bd-439f-8e63-28c9fc8b32e5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.339236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 903/1797] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:03 2022] PUT /identity/v3/projects/fd695a014b6c4f07845cc455545feb2d/users/cf11538858014581989eec80d13b9479/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:03.351374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.351791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.352106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.355543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 895/1798] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:03 2022] PUT /identity/v3/projects/774007a9c4384a0ab719ed271e9873d5/users/a980255e293e415b997dac71fb8e99f0/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:03.369560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.369935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.370440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.416796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] MFA Rules not processed for user `cf11538858014581989eec80d13b9479`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:03.418293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] MFA Rules not processed for user `a980255e293e415b997dac71fb8e99f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:03.420401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:03.421101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:03.530983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 904/1799] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:03.549408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.552508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:03.559556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 896/1800] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:03.573719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.575394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:03.638447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:03.656563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e8360-6398-4d2f-aba9-fe1c12cea486 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.660879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.662013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.662265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.662668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.686385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.688030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.699927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d84d7-8fc5-4c53-8b74-eed49cabd87d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.708963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.712924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.713322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.714645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.724329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 897/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:03.755648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:03.764234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:03.794067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 905/1802] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:29:03.906381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.924720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176710-bdea-44db-bb34-9d92f110177d service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.955434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.958660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:03.971882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.971882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.973223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:03.977662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495623-9f82-49ec-9109-d07e49aae41e service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:03.985140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:03.985936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:03.986553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:03.986839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:04.012833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:04.070419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:04.071586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:04.073084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:04.077573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:04.089580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 898/1803] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:04.101264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 906/1804] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:29:06.054412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:06.069294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e5d3-89e9-4c6b-afa9-45bda4566945 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:06.073740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:06.074453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:06.074815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:06.075387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:06.079997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:06.214872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:06.216392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72af871-f19d-401e-9ffb-dbe707851b9f service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:06.236342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 899/1805] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:07.930319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:07.931809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:07.996481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770a61d-eb10-4ba1-9fa2-b5fcea67cea5 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:08.001062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:08.001829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:08.002112 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:08.002382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] PATH_INFO: `/v3/users/d6ea79543341478389dd108c2240168d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:08.003554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ea79543341478389dd108c2240168d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:08.006055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:08.052119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ea79543341478389dd108c2240168d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.052822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ea79543341478389dd108c2240168d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.063243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ea79543341478389dd108c2240168d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.071512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ea79543341478389dd108c2240168d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.083656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:08.086488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ea79543341478389dd108c2240168d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.094766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 907/1806] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:07 2022] DELETE /identity/v3/users/d6ea79543341478389dd108c2240168d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:08.099123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:08.100078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:08.100362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:08.119695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:08.120994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:08.181301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 900/1807] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:08.190504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:08.191949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:08.248381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b44047-cd50-42d9-8fda-10e981ccb92f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:08.251943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:08.252556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:08.252839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:08.253115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:08.262669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:08.264119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:08.273357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 908/1808] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:08.540121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:08.541565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:08.556129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9459e81-5a6a-45c6-b1b7-a1b510a4667a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:08.561743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:08.562335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:08.562645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:08.562987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] PATH_INFO: `/v3/projects/afa19fcb601f475f9c9478b5084701d5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:08.564120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa19fcb601f475f9c9478b5084701d5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:08.565996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:08.612689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa19fcb601f475f9c9478b5084701d5', 'request_id': 'req-e373139a-fe65-4347-aa09-d96c367a84ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.614651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:08.619395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa19fcb601f475f9c9478b5084701d5', 'request_id': 'req-e373139a-fe65-4347-aa09-d96c367a84ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.629599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa19fcb601f475f9c9478b5084701d5', 'request_id': 'req-e373139a-fe65-4347-aa09-d96c367a84ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.643305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] The token cache is being invalidate because project afa19fcb601f475f9c9478b5084701d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:08.643604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.644306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.647108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 901/1809] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:08 2022] DELETE /identity/v3/projects/afa19fcb601f475f9c9478b5084701d5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:08.650996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:08.663316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60c79-f8ac-4220-9dd2-acdf2ec1eeba service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:08.732650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:08.733327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:08.733594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:08.733869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] PATH_INFO: `/v3/users/fea34bb8087d4db692174551c9273f99` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:08.735154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea34bb8087d4db692174551c9273f99)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:08.737336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:08.785539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea34bb8087d4db692174551c9273f99'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.786300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea34bb8087d4db692174551c9273f99'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.794255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea34bb8087d4db692174551c9273f99'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.802128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea34bb8087d4db692174551c9273f99'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.814048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c49f946d-9e64-427e-a0e3-5e1d16ce4263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea34bb8087d4db692174551c9273f99'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:08.820891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 909/1810] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:08 2022] DELETE /identity/v3/users/fea34bb8087d4db692174551c9273f99 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:09.133156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.134756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:09.148410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373139a-fe65-4347-aa09-d96c367a84ab admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.210491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.211274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.211669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.212088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] PATH_INFO: `/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.213321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6715f22d6c5c4afb825c2b9b0be9ac1e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.215347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.271951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6715f22d6c5c4afb825c2b9b0be9ac1e', 'request_id': 'req-3f0b5be3-fa41-4f49-97a2-6498e7248833'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:09.278007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6715f22d6c5c4afb825c2b9b0be9ac1e', 'request_id': 'req-3f0b5be3-fa41-4f49-97a2-6498e7248833'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:09.287899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6715f22d6c5c4afb825c2b9b0be9ac1e', 'request_id': 'req-3f0b5be3-fa41-4f49-97a2-6498e7248833'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:09.299518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] The token cache is being invalidate because project 6715f22d6c5c4afb825c2b9b0be9ac1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:09.299807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:09.300504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3f0b5be3-fa41-4f49-97a2-6498e7248833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:09.302700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 902/1811] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:09 2022] DELETE /identity/v3/projects/6715f22d6c5c4afb825c2b9b0be9ac1e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:09.307908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.308374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.308588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.330222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:09.331425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:09.341280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:09.402528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 910/1812] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:09.411980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.412298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.412575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.432260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:09.433452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:09.441192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 903/1813] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:09.445486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.455330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe44dc-e62b-49c0-88c9-30f4cc2457b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.460427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.461037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.461317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.461585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.462278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:09.462575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.464292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.478520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 911/1814] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:09.482878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.492455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc37c89-ff87-490f-bcc4-c648bd3d0986 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.497238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.497851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.498138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.498417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.499095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.501035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.532896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 904/1815] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:09 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:09.534103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.545742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b60b-b755-4b7d-9d54-4252d1936c89 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.602931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.603544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.603848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.604124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.604770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.606661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.627297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 912/1816] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:09 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:09.631480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.641846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3a63-e28a-479a-ab97-f9a137d2d3ef admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.646992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.647600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.647924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.648209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.648937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.650538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.664304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 905/1817] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:09.668237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.679007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a747-6496-468e-90f9-53fc9b5aedaa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.684004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.684610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.684988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.685316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.686006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.687582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f77a99-2ccd-41d3-b486-055ad3f42e40 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.704763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 913/1818] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:09.711086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:09.729267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24972c2a-82c5-4c2d-baca-5d008a6f8bde admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:09.734468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:09.735156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.735474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.735870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] PATH_INFO: `/v3/projects/f48f22b6a61d4fcca5e546c911dea185/users/33991a1b7f2d4386ad10442ba631a5eb/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.771898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48f22b6a61d4fcca5e546c911dea185, user_id=33991a1b7f2d4386ad10442ba631a5eb, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:09.775861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:09.809140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 906/1819] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:09 2022] PUT /identity/v3/projects/f48f22b6a61d4fcca5e546c911dea185/users/33991a1b7f2d4386ad10442ba631a5eb/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:09.811052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:09.811394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:09.811674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:09.891923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] MFA Rules not processed for user `33991a1b7f2d4386ad10442ba631a5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:09.893277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:10.014635 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 914/1820] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:10.018857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.037536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:10.110298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e08968-8cae-4de0-80cd-144f8c75cac2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.115220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.116013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.116368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.116714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.117473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.119486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71074edf-81fd-4804-b755-011fe354b6bf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.157262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 907/1821] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:10 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:10.161481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.172940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e19ed-04d6-4408-b236-7ac8c49906b0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.179553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706147b-3f14-4d93-b9d5-db0dc04f3269 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.180387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706147b-3f14-4d93-b9d5-db0dc04f3269 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.180669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706147b-3f14-4d93-b9d5-db0dc04f3269 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.182179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 908/1822] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:29:10.187159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:10.194330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.194615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.194888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.218719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:10.220052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:10.277064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.277844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.278033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.278316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.279008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.280972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.307789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 915/1823] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:10 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:10.311524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.316233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 909/1824] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:29:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:10.327106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3860f23-c567-4557-8eed-ac4b7fc9ba97 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.332299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.333688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:10.341290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.342164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.342715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.343128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.345493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.347729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.348289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22e394-a7f5-4101-a342-5d98cc9bc70e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.354342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.354962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.355253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.355509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.359678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 916/1825] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:29:10.363573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.368591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.370197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.374248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e90f-3803-4540-8bfb-eb8456fccb71 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.382215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 910/1826] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:10.383993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.385014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.385303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.386126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] PATH_INFO: `/v3/projects/66da08535a5b4c79b02622a4294406b4/users/8899ddb1f1384ad19d6f34099fa329ba/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.403289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=66da08535a5b4c79b02622a4294406b4, user_id=8899ddb1f1384ad19d6f34099fa329ba, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.405508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.421055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 917/1827] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:10 2022] PUT /identity/v3/projects/66da08535a5b4c79b02622a4294406b4/users/8899ddb1f1384ad19d6f34099fa329ba/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:10.426225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.439594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496c5cb-dec9-48ea-b109-06d202fe9093 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.503085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.503714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.504035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.504378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.505654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.507261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a834d8-75ac-4df5-a4b5-becc29cca61a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.518051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 911/1828] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:10.522157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.532346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c13f-2bf1-46e1-b0e8-0d1ac65f3aca admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.536923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.537588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.537877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.538156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] PATH_INFO: `/v3/projects/66da08535a5b4c79b02622a4294406b4/users/8899ddb1f1384ad19d6f34099fa329ba/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.540183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] RBAC: Authorizing `identity:create_grant(project_id=66da08535a5b4c79b02622a4294406b4, user_id=8899ddb1f1384ad19d6f34099fa329ba, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.542816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.565196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 918/1829] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:10 2022] PUT /identity/v3/projects/66da08535a5b4c79b02622a4294406b4/users/8899ddb1f1384ad19d6f34099fa329ba/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:10.566675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.566972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.567858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.583803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:10.599923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] MFA Rules not processed for user `8899ddb1f1384ad19d6f34099fa329ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:10.601145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:10.674042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 912/1830] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:10.685767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.687563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:10.745125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0fa5-f702-4e14-abd5-726224ad46db admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.748750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.749360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.749666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.749941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.758855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.760321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.769604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 919/1831] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:10.873030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:10.882238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11a8b-3fd1-44d2-afed-e1b6dc9fd3ba None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:10.885811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:10.886419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:10.886740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:10.887047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:10.950035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:10.951491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def90c-7997-4814-8ab2-4a21359820d5 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:10.961746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 913/1832] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:11.267272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:11.324208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:11.329859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8641c-23e4-4e8c-9910-d357bde92160 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:11.335123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:11.335711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:11.336027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:11.336304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] PATH_INFO: `/v3/users/33991a1b7f2d4386ad10442ba631a5eb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:11.337453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33991a1b7f2d4386ad10442ba631a5eb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:11.339348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:11.382704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33991a1b7f2d4386ad10442ba631a5eb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:11.383353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33991a1b7f2d4386ad10442ba631a5eb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:11.390070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33991a1b7f2d4386ad10442ba631a5eb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:11.396807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33991a1b7f2d4386ad10442ba631a5eb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:11.408259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33991a1b7f2d4386ad10442ba631a5eb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:11.414780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 920/1833] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:11 2022] DELETE /identity/v3/users/33991a1b7f2d4386ad10442ba631a5eb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:11.419191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:11.419621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:11.419816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:11.439126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:11.440348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:11.502813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 914/1834] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:11.512606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:11.514307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:11.573995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b6005-3752-4209-922e-3be2466e8c81 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:11.578098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:11.578902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:11.579409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:11.579946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:11.591504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:11.593008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:11.603081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 921/1835] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:11.958330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:11.959868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:11.976939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70f8dc-8086-42c3-9c94-b9477ee29d20 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:11.982296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:11.982965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:11.983221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:11.983481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] PATH_INFO: `/v3/projects/f48f22b6a61d4fcca5e546c911dea185` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:11.984864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48f22b6a61d4fcca5e546c911dea185)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:11.986791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:12.016951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:12.035092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48f22b6a61d4fcca5e546c911dea185', 'request_id': 'req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.041237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48f22b6a61d4fcca5e546c911dea185', 'request_id': 'req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.049720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48f22b6a61d4fcca5e546c911dea185', 'request_id': 'req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.061247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] The token cache is being invalidate because project f48f22b6a61d4fcca5e546c911dea185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:12.061561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.062195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.063808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 915/1836] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:11 2022] DELETE /identity/v3/projects/f48f22b6a61d4fcca5e546c911dea185 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:12.068679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:12.079116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8477a-9d38-48b6-be84-18e4e8009b8f service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:12.134940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:12.135525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:12.135807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:12.136120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] PATH_INFO: `/v3/users/8899ddb1f1384ad19d6f34099fa329ba` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:12.137309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8899ddb1f1384ad19d6f34099fa329ba)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:12.139208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:12.159937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:12.185141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8899ddb1f1384ad19d6f34099fa329ba'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.185805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8899ddb1f1384ad19d6f34099fa329ba'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.196989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8899ddb1f1384ad19d6f34099fa329ba'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.204880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8899ddb1f1384ad19d6f34099fa329ba'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.217875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-421bcff9-0922-4335-8939-181cb9b56344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8899ddb1f1384ad19d6f34099fa329ba'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.223851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 922/1837] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:12 2022] DELETE /identity/v3/users/8899ddb1f1384ad19d6f34099fa329ba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:12.492554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:12.494060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:12.508034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb8f35-2d8c-485f-a129-5db7ef0f84cd admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:12.564212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:12.564855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:12.565134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:12.565405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] PATH_INFO: `/v3/projects/66da08535a5b4c79b02622a4294406b4` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:12.566564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66da08535a5b4c79b02622a4294406b4)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:12.569313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:12.615691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66da08535a5b4c79b02622a4294406b4', 'request_id': 'req-75e77cea-21a9-4662-9e5c-55b15e109d75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.621161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66da08535a5b4c79b02622a4294406b4', 'request_id': 'req-75e77cea-21a9-4662-9e5c-55b15e109d75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.628558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66da08535a5b4c79b02622a4294406b4', 'request_id': 'req-75e77cea-21a9-4662-9e5c-55b15e109d75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.640376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] The token cache is being invalidate because project 66da08535a5b4c79b02622a4294406b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:12.640658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.641302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75e77cea-21a9-4662-9e5c-55b15e109d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:12.642823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 916/1838] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:12 2022] DELETE /identity/v3/projects/66da08535a5b4c79b02622a4294406b4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:12.647671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:12.648213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:12.648302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:12.671304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:12.672481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:12.803051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 923/1839] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:12.813255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:12.813542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:12.813887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:12.862304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:12.867636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:12.894190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:12.895242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 917/1840] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:12.912545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6121fda-83a3-4949-9b0d-8278c8155c6b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:12.925154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:12.927603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:12.929370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:12.929673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:12.930437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:12.930807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:12.933446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:12.975907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 924/1841] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:12.980800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:12.995892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a15f1-575a-4019-a1b8-ca66d87f633e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.006960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.009887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.010591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.011395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.011650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.013866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.083250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 918/1842] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:12 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:13.084189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:13.105074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d73bef-6f5d-4b05-9227-7d8c37237403 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.338600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.339293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.339932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.340242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.340927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.342999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.427965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 925/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:13 2022] POST /identity/v3/users => generated 289 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:13.431560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:13.449056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3faba-8d4f-4278-8bd8-5393b4558f4e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.461438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.462170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.462540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.462917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.464694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.468476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.499180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 919/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:13.503319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:13.519277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562b9f-5bfa-41e8-84f0-60e29c0dbcb6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.537086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.538224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.538519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.538801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.541709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.545222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d71326-2487-4a33-aad5-b4f1fb96930e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.575810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 926/1845] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:13.579957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:13.587585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:13.597218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b88cb-ce14-46a1-b472-8b1f687ed6c4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.609474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.610547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.611199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.611694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] PATH_INFO: `/v3/projects/820dc044083246338c9c70b890ffd920/users/297d730bc6154e4ba3464de81811f998/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.660810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820dc044083246338c9c70b890ffd920, user_id=297d730bc6154e4ba3464de81811f998, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.663986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.690749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 920/1846] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:13 2022] PUT /identity/v3/projects/820dc044083246338c9c70b890ffd920/users/297d730bc6154e4ba3464de81811f998/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:13.701832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.702298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.702385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.711814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:13.758504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] MFA Rules not processed for user `297d730bc6154e4ba3464de81811f998`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:13.762487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:13.869549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 927/1847] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:13.882393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:13.885003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:13.956574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d394-e524-4967-862c-6bb548ca8cc3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:13.970548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:13.971281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:13.971635 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:13.972554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:13.982324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:13.984333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51163f-01ff-4c8a-9d0d-c136680e5d84 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:13.996998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 921/1848] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:14.454016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:14.455585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:14.540011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729dc53-8af5-4bf5-ad30-c4741782fd45 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:14.544592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:14.545236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:14.545518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:14.545796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] PATH_INFO: `/v3/users/44d53ede50424e0492def4d530c7ca76` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:14.547149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d53ede50424e0492def4d530c7ca76)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:14.549279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:14.601678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d53ede50424e0492def4d530c7ca76'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:14.602335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d53ede50424e0492def4d530c7ca76'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:14.613019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d53ede50424e0492def4d530c7ca76'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:14.621621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d53ede50424e0492def4d530c7ca76'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:14.634149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d53ede50424e0492def4d530c7ca76'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:14.640353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 928/1849] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:14 2022] DELETE /identity/v3/users/44d53ede50424e0492def4d530c7ca76 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:14.644854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:14.645345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:14.645449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:14.664393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:14.665584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:14.724700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 922/1850] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:14.735241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:14.736958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:14.751848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:14.808668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cb70-c743-4e98-a639-c9e196605f6b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:14.812828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:14.813834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:14.814300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:14.814753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:14.825763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:14.827591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:14.839522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 929/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:15.118975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:15.120539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:15.133796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8fc03-88cf-4691-82ad-976bd38c232d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:15.138691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:15.139460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:15.139867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:15.140210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] PATH_INFO: `/v3/projects/8c8e0d1e0dae4ce2949c98021480e5fa` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:15.141491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c8e0d1e0dae4ce2949c98021480e5fa)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:15.143516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:15.196274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c8e0d1e0dae4ce2949c98021480e5fa', 'request_id': 'req-a1feac91-a600-451f-82f8-4a84f6a3896d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.203803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c8e0d1e0dae4ce2949c98021480e5fa', 'request_id': 'req-a1feac91-a600-451f-82f8-4a84f6a3896d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.213966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c8e0d1e0dae4ce2949c98021480e5fa', 'request_id': 'req-a1feac91-a600-451f-82f8-4a84f6a3896d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.227592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] The token cache is being invalidate because project 8c8e0d1e0dae4ce2949c98021480e5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:15.227923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.228776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.230790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 923/1852] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:15 2022] DELETE /identity/v3/projects/8c8e0d1e0dae4ce2949c98021480e5fa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:15.235021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:15.251937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff751b6-6d5d-4039-9c8f-62e69de8a21d service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:15.334921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:15.335675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:15.336056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:15.336364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] PATH_INFO: `/v3/users/cf11538858014581989eec80d13b9479` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:15.337591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf11538858014581989eec80d13b9479)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:15.339510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:15.385627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf11538858014581989eec80d13b9479'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.386275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf11538858014581989eec80d13b9479'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.393436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf11538858014581989eec80d13b9479'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.400859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf11538858014581989eec80d13b9479'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.412495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-553d9c76-2c2c-4327-828b-6ae6708ff2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf11538858014581989eec80d13b9479'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.418258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 930/1853] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:15 2022] DELETE /identity/v3/users/cf11538858014581989eec80d13b9479 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:15.726203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:15.727639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:15.744169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feac91-a600-451f-82f8-4a84f6a3896d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:15.747943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:15.805647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:15.806288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:15.806705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:15.806837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] PATH_INFO: `/v3/projects/fd695a014b6c4f07845cc455545feb2d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:15.808012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd695a014b6c4f07845cc455545feb2d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:15.809914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:15.855171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd695a014b6c4f07845cc455545feb2d', 'request_id': 'req-ecf9375a-dbd8-4fdb-8474-893aeb38107b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.860860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd695a014b6c4f07845cc455545feb2d', 'request_id': 'req-ecf9375a-dbd8-4fdb-8474-893aeb38107b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.870873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd695a014b6c4f07845cc455545feb2d', 'request_id': 'req-ecf9375a-dbd8-4fdb-8474-893aeb38107b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.882208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] The token cache is being invalidate because project fd695a014b6c4f07845cc455545feb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:15.882491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.883147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ecf9375a-dbd8-4fdb-8474-893aeb38107b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:15.884709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 924/1854] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:15 2022] DELETE /identity/v3/projects/fd695a014b6c4f07845cc455545feb2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:15.890703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:15.891169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:15.891259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:15.911614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:15.912841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:15.942561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:15.976944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 931/1855] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:15.984791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:15.985147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:15.985499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.005330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:16.006527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:16.014466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 925/1856] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:16.018645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.028202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0482928-3c74-430a-be53-826b391d35d4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.032647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.033249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.033524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.033825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.034468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:16.034767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.036930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.048133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 932/1857] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:29:16.052763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.062576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dbaa2-df09-434c-b061-f6254d199310 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.067867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.068644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.069211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.069525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.070225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.072151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.099703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 926/1858] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:16.103810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.113485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415967f5-8825-45bc-a09e-a64a7f6f2a76 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.177907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.178529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.178812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.179088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.179763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.181694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.204709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 933/1859] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:16.209242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.219401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3354c9-ed3a-4454-bd9f-5fad6645fe2a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.224582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.225214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.225493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.225769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.226461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.228100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.237815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 927/1860] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:16.241660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.251359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881a194-e267-4c89-b08e-0a3028cb4dd2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.255856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.256466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.256748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.257023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.257706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.259274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d67feb-1468-47a9-aa8d-7784741ed279 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.271513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 934/1861] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:29:16.274490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.284984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0adf4-d161-4cb5-829d-4519c02568b7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.290393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.291165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.291452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.291733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] PATH_INFO: `/v3/projects/d773ef9f0bff4c44b8d0c2cf709040e8/users/c59de041a4f94511be33678a77b3c4fc/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.307215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d773ef9f0bff4c44b8d0c2cf709040e8, user_id=c59de041a4f94511be33678a77b3c4fc, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.309911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.323166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 928/1862] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:16 2022] PUT /identity/v3/projects/d773ef9f0bff4c44b8d0c2cf709040e8/users/c59de041a4f94511be33678a77b3c4fc/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:16.334157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.334573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.334920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.363674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] MFA Rules not processed for user `c59de041a4f94511be33678a77b3c4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:16.364943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:16.439116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 935/1863] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:16.443614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.480146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:16.510146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b4a59-d802-457a-99e7-4d9295c81b7d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.514786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.515401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.515686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.515990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.516654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.518552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.549838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 929/1864] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:16.554056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.565089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cd5a7-551f-4ace-8c90-3d2f10b02fdd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.593727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:16.628389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.629012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.629860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.630133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.630807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.632743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.654257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 936/1865] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:16.658431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.668753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc282c-1cbb-4614-8745-ee52b6363015 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.673372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.673980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.674261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.674536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.675227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.676888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.687623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 930/1866] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:16.691728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.702885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e098b3-bcce-46e2-a6f0-b2ca28b9465a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.708155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.708826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.709111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.709392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] PATH_INFO: `/v3/projects/0f416391d4a84663954efd8684b6fd42/users/702f0a30ebfc46abb8d0166db234ac60/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.726438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f416391d4a84663954efd8684b6fd42, user_id=702f0a30ebfc46abb8d0166db234ac60, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.729171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.746478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 937/1867] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:16 2022] PUT /identity/v3/projects/0f416391d4a84663954efd8684b6fd42/users/702f0a30ebfc46abb8d0166db234ac60/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:16.750835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.762242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3684e4-4859-415b-bd37-87fe96d07aa5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.828751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.829371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.829652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.829926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.830615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.832259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376d74-ee99-4c10-b6cc-e31a015d19f6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.842405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 931/1868] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:29:16.846544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:16.856917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1df41-9283-4e68-93e0-537211afd9a2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:16.861523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:16.862175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.862454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.862728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] PATH_INFO: `/v3/projects/0f416391d4a84663954efd8684b6fd42/users/702f0a30ebfc46abb8d0166db234ac60/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.864684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f416391d4a84663954efd8684b6fd42, user_id=702f0a30ebfc46abb8d0166db234ac60, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:16.867361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:16.877673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 938/1869] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:16 2022] PUT /identity/v3/projects/0f416391d4a84663954efd8684b6fd42/users/702f0a30ebfc46abb8d0166db234ac60/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:16.889939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:16.890230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:16.890511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:16.937281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] MFA Rules not processed for user `702f0a30ebfc46abb8d0166db234ac60`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:16.938543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:17.050869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 932/1870] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:17.065647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:17.069559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:17.164526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361366c-fdce-4c82-8fef-424e5321eddf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:17.168733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:17.169403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:17.169930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:17.170144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:17.213996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:17.216462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:17.235387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 939/1871] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:17.346192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:17.356944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec034a0-61d2-4a01-9ac9-c8dce1abf8ab None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:17.360735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:17.361478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:17.362030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:17.362188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:17.390832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:17.422783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:17.424378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b78061-c5a5-4f92-a812-d01168e06d7b service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:17.434811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 933/1872] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:22.892114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:22.893574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:22.976150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec9490-569e-4ed9-8549-aa8413189191 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:22.980854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:22.981500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:22.981795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:22.982066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] PATH_INFO: `/v3/users/a106467c95da4254b15875c3c639552f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:22.983223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a106467c95da4254b15875c3c639552f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:22.985152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:23.017196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:23.058413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a106467c95da4254b15875c3c639552f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.059096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a106467c95da4254b15875c3c639552f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.070843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a106467c95da4254b15875c3c639552f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.079093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a106467c95da4254b15875c3c639552f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.093833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a106467c95da4254b15875c3c639552f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.101994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 940/1873] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:22 2022] DELETE /identity/v3/users/a106467c95da4254b15875c3c639552f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:23.106456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:23.106973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:23.107053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:23.137704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:23.138961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:23.227480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 934/1874] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:23.237587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:23.239714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:23.317588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836244e-bcbd-464f-9e3f-caec64588d08 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:23.321497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:23.322114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:23.322392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:23.322664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:23.331673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:23.333683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:23.343603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 941/1875] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:23.739064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:23.740748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:23.764508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cf035-27d5-40eb-a16d-b6537ff15bf9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:23.782835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:23.783710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:23.784216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:23.784667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] PATH_INFO: `/v3/projects/de92eeb0d263464ea91f5d04f7f8c0d0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:23.786400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de92eeb0d263464ea91f5d04f7f8c0d0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:23.788832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:23.877047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de92eeb0d263464ea91f5d04f7f8c0d0', 'request_id': 'req-c8a297a4-496c-4755-9abf-c0b8a967cc33'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.885003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de92eeb0d263464ea91f5d04f7f8c0d0', 'request_id': 'req-c8a297a4-496c-4755-9abf-c0b8a967cc33'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.902496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de92eeb0d263464ea91f5d04f7f8c0d0', 'request_id': 'req-c8a297a4-496c-4755-9abf-c0b8a967cc33'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.925832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] The token cache is being invalidate because project de92eeb0d263464ea91f5d04f7f8c0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:23.926599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.927278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:23.929772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 935/1876] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:23 2022] DELETE /identity/v3/projects/de92eeb0d263464ea91f5d04f7f8c0d0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:23.932905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:23.944370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecc713-00cb-4609-ad2f-6f0cab9e86de service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.012856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.014066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.014066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.014066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] PATH_INFO: `/v3/users/a980255e293e415b997dac71fb8e99f0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.015539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a980255e293e415b997dac71fb8e99f0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.017697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.070656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a980255e293e415b997dac71fb8e99f0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.071538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a980255e293e415b997dac71fb8e99f0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.073521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:24.082231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a980255e293e415b997dac71fb8e99f0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.092441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a980255e293e415b997dac71fb8e99f0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.106135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6f67e69b-01b1-4224-9ed4-519b0ce3db60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a980255e293e415b997dac71fb8e99f0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.112720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 942/1877] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:23 2022] DELETE /identity/v3/users/a980255e293e415b997dac71fb8e99f0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:24.384829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.386479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:24.403579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a297a4-496c-4755-9abf-c0b8a967cc33 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.451523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:24.463106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.463945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.464522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.464803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] PATH_INFO: `/v3/projects/774007a9c4384a0ab719ed271e9873d5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.466044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774007a9c4384a0ab719ed271e9873d5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.468002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.512986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774007a9c4384a0ab719ed271e9873d5', 'request_id': 'req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.518328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774007a9c4384a0ab719ed271e9873d5', 'request_id': 'req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.525708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774007a9c4384a0ab719ed271e9873d5', 'request_id': 'req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.537011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] The token cache is being invalidate because project 774007a9c4384a0ab719ed271e9873d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:24.537293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.537918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b92a5e-14cb-44d9-98d6-f8bf0a2c8330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:24.539412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 936/1878] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:24 2022] DELETE /identity/v3/projects/774007a9c4384a0ab719ed271e9873d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:24.545199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.545487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.545762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.564927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:24.566124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:24.632440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 943/1879] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:24.640410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.640699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.640980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.662877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:24.664131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:24.672194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 937/1880] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:24.677133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.687170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2806be4-50c2-42cd-9269-23303422da3d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.694098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.694721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.694996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.695269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.696009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:24.696314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.697912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.711194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 944/1881] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:24.715632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.725756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d832a2f-de14-4688-b157-c56aecf955cd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.730865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.731480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.731759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.732064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.732730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.734762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.767875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 938/1882] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:24 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:24.773307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.784490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa78ff-f9fd-4987-8427-2b6e0b664e76 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.842275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.842889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.843172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.843449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.844146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.846053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.865975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 945/1883] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:24 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:24.870137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.880942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e759468-d61e-4e8a-8247-5c673d89d60e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.885711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.886324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.886604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.886878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.887571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.889278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.900182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 939/1884] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:24.903884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.913581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f38-da4f-4ab3-8d5b-d5e125112af7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.918140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.918739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.919018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.919292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.920017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.921608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa871a-edb8-4963-b0d3-f22eb95f05de admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.932499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 946/1885] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:24.936730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:24.947741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b022127-520a-4fb0-9f0d-b7f4de2a4556 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:24.956987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:24.958243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:24.958525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:24.958921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] PATH_INFO: `/v3/projects/ce154589d8bd48e8ab445ad28c7dbfdd/users/a1b78f227cfe432f948473f915a8b6ab/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:24.980904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce154589d8bd48e8ab445ad28c7dbfdd, user_id=a1b78f227cfe432f948473f915a8b6ab, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:24.983943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:24.996676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 940/1886] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:24 2022] PUT /identity/v3/projects/ce154589d8bd48e8ab445ad28c7dbfdd/users/a1b78f227cfe432f948473f915a8b6ab/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:25.008802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:25.009323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:25.009605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:25.037686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] MFA Rules not processed for user `a1b78f227cfe432f948473f915a8b6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:25.038930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:25.057656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:25.108977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 947/1887] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:25.119890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:25.121633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:25.210372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf08b36-39d5-47a4-b3b1-78abe80711a0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:25.214607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:25.217375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:25.217375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:25.217375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:25.226315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:25.227676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf2964-6f5c-480b-b0c7-6f7d7cca9520 service glance] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:25.234297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:25.242105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 941/1888] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:30.081265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:30.082967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:30.183718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790227-bf09-4b65-a0a1-59531479332c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:30.195602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:30.197365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:30.197662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:30.197989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] PATH_INFO: `/v3/users/c59de041a4f94511be33678a77b3c4fc` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:30.203869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59de041a4f94511be33678a77b3c4fc)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:30.208008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:30.321494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59de041a4f94511be33678a77b3c4fc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:30.322192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59de041a4f94511be33678a77b3c4fc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:30.333752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59de041a4f94511be33678a77b3c4fc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:30.344778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59de041a4f94511be33678a77b3c4fc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:30.383795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59de041a4f94511be33678a77b3c4fc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:30.399422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 948/1889] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:30 2022] DELETE /identity/v3/users/c59de041a4f94511be33678a77b3c4fc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:30.400891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:30.401497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:30.401497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:30.430026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:30.431663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:30.511441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 942/1890] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:30.523440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:30.525631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:30.583221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:30.600205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2bf97-8707-4b94-a58b-6bab32e9e8a1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:30.603888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:30.604509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:30.604807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:30.605079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:30.615265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:30.617240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:30.631279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 949/1891] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:31.069106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:31.070841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:31.085224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bf6a8-15f9-4c36-a620-f5d7fbfb6c9e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:31.091318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:31.092192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:31.092581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:31.092937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] PATH_INFO: `/v3/projects/d773ef9f0bff4c44b8d0c2cf709040e8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:31.097089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d773ef9f0bff4c44b8d0c2cf709040e8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:31.113894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:31.184826 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd773ef9f0bff4c44b8d0c2cf709040e8', 'request_id': 'req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.193790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd773ef9f0bff4c44b8d0c2cf709040e8', 'request_id': 'req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.203030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd773ef9f0bff4c44b8d0c2cf709040e8', 'request_id': 'req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.222197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] The token cache is being invalidate because project d773ef9f0bff4c44b8d0c2cf709040e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:31.222553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.223850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.225436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 943/1892] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:31 2022] DELETE /identity/v3/projects/d773ef9f0bff4c44b8d0c2cf709040e8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:31.229392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:31.242309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdf2d-a1ce-4160-b67a-6415fd563a36 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:31.328300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:31.328948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:31.329220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:31.329492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] PATH_INFO: `/v3/users/702f0a30ebfc46abb8d0166db234ac60` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:31.330643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=702f0a30ebfc46abb8d0166db234ac60)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:31.334806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:31.402665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702f0a30ebfc46abb8d0166db234ac60'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.403437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702f0a30ebfc46abb8d0166db234ac60'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.412313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702f0a30ebfc46abb8d0166db234ac60'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.420740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702f0a30ebfc46abb8d0166db234ac60'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.436939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-53ac036e-78bd-48ed-8cf6-c338a93e22dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702f0a30ebfc46abb8d0166db234ac60'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.458622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 950/1893] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:31 2022] DELETE /identity/v3/users/702f0a30ebfc46abb8d0166db234ac60 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:31.852966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:31.854772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:31.870007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ab30-b1b9-4f96-93f2-6ec1d8985b4b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:31.930078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:31.930800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:31.931195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:31.931534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] PATH_INFO: `/v3/projects/0f416391d4a84663954efd8684b6fd42` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:31.932804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f416391d4a84663954efd8684b6fd42)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:31.934986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:31.968175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:31.985214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f416391d4a84663954efd8684b6fd42', 'request_id': 'req-f51c3c2a-2358-435c-96e0-085c267d390c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.991111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f416391d4a84663954efd8684b6fd42', 'request_id': 'req-f51c3c2a-2358-435c-96e0-085c267d390c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:31.998338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f416391d4a84663954efd8684b6fd42', 'request_id': 'req-f51c3c2a-2358-435c-96e0-085c267d390c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.009727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] The token cache is being invalidate because project 0f416391d4a84663954efd8684b6fd42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:32.010009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.010631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f51c3c2a-2358-435c-96e0-085c267d390c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.012166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 944/1894] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:31 2022] DELETE /identity/v3/projects/0f416391d4a84663954efd8684b6fd42 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:32.017074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.017362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.017640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.036725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:32.037918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.098081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 951/1895] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:32.105891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.106176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.106453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.125964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:32.127148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.135483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 945/1896] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:32.139489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.144328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:32.150911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cbb70-dd09-4dc2-a360-60da3e59cb91 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.156760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.157402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.157681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.157949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.158639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:32.158941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.160592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.172359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 952/1897] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:32.177520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.186964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a02563-529a-4380-9ab8-bc9afc6811aa None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.199972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.199972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.199972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.199972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.200311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.200311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.224226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 946/1898] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:32.228380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.239041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b51a6-1063-4638-a268-5f2e0727a365 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.293828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.294472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.294757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.295039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.295709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.297689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.319235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 953/1899] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:32.323413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.333733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059c5d8-3119-461b-8a4f-6e38f5287dc8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.338247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.338852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.339132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.339406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.340121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.341748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.352651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 947/1900] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:32.356663 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.367667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ac58-3dd4-40f7-a094-7e9f790d2f03 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.372546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.373168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.373466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.373724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.374415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.376038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74b41-be12-4b9f-b8ce-56ecb9332675 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.387130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 954/1901] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:32.391310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.401496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fa85-3e9a-4426-abeb-7f8ad89a4a6b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.406117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.406749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.407033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.407311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] PATH_INFO: `/v3/projects/67ab99bda5014f9d9712515104bd57bc/users/4d79a54b3f6645bf995266dd0aee49ef/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.422607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ab99bda5014f9d9712515104bd57bc, user_id=4d79a54b3f6645bf995266dd0aee49ef, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.425315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.440042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 948/1902] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:32 2022] PUT /identity/v3/projects/67ab99bda5014f9d9712515104bd57bc/users/4d79a54b3f6645bf995266dd0aee49ef/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:32.451432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.451727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.452033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.480344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] MFA Rules not processed for user `4d79a54b3f6645bf995266dd0aee49ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:32.481543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.545533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 955/1903] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:32.559429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.561338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.618797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22dca-801f-4ace-b82f-2dfa03702406 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.622411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.623022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.623304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.623582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.632036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.633548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6c113-2c1a-4d4c-9484-7d200d6c278f service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.642552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 949/1904] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:32.661688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.719993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:32.724801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed2c79-638c-47f0-baa2-bf83210701a8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.730095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.730734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.731017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.731296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] PATH_INFO: `/v3/users/4d79a54b3f6645bf995266dd0aee49ef` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.732513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d79a54b3f6645bf995266dd0aee49ef)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.734457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:32.778502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d79a54b3f6645bf995266dd0aee49ef'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.779158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d79a54b3f6645bf995266dd0aee49ef'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.786588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d79a54b3f6645bf995266dd0aee49ef'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.793668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d79a54b3f6645bf995266dd0aee49ef'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.805240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d79a54b3f6645bf995266dd0aee49ef'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:32.811072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 956/1905] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:32 2022] DELETE /identity/v3/users/4d79a54b3f6645bf995266dd0aee49ef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:32.815654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.816007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.816409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.836168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:32.837394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.884535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:32.899456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 950/1906] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:32.909083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:32.911073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:32.981454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7506e-a38f-4afc-a844-9c14c1d189f6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:32.985267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:32.985982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:32.986339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:32.986685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:32.996443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:32.997872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf0ad-7a66-414a-89ee-62e907935a02 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.007750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 957/1907] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:33.354755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.356320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:33.368445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c70948-9c34-4a5d-beed-f633c6515c5e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.373089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.373848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.374129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.374738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] PATH_INFO: `/v3/projects/67ab99bda5014f9d9712515104bd57bc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.375592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ab99bda5014f9d9712515104bd57bc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.377541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.430288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ab99bda5014f9d9712515104bd57bc', 'request_id': 'req-9cb032bd-b81a-4938-b918-f63cc4c257f2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:33.435558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ab99bda5014f9d9712515104bd57bc', 'request_id': 'req-9cb032bd-b81a-4938-b918-f63cc4c257f2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:33.442895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ab99bda5014f9d9712515104bd57bc', 'request_id': 'req-9cb032bd-b81a-4938-b918-f63cc4c257f2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:33.453990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] The token cache is being invalidate because project 67ab99bda5014f9d9712515104bd57bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:33.454278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:33.454971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9cb032bd-b81a-4938-b918-f63cc4c257f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:33.456571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 951/1908] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:33 2022] DELETE /identity/v3/projects/67ab99bda5014f9d9712515104bd57bc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:33.462274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.462767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.462858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.481950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:33.483143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:33.541610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 958/1909] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:33.549414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.549691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.549970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.587326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:33.588579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:33.607867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 952/1910] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:33.610628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.625228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc86ed-c208-4f4d-9827-a5632754b53f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.630589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.631237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.631516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.631787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.632512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:33.632814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.634409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.646273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 959/1911] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:33.650617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.660553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334653e-5b1e-48b2-85cf-8c764c191d23 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.665073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.665685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.665963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.666250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.666937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.668858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.698938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 953/1912] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:33 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:33.703349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.713336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44178cef-80f7-45c5-a400-557ecb33a089 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.716830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:33.772304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.772925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.773207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.773486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.774160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.776095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.797975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 960/1913] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:33 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:33.802736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.812995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc8141-2d6d-4e46-ac4b-039b2a69fca5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.818326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.819188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.819466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.819741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.820461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.822060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.832811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 954/1914] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:33.837041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.847106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7729eda-06d3-4300-90f0-b1a5c7567319 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.852568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.853288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.853674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.853959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.854816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.856456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac8e9c-0637-4c6e-8516-a55633f947b9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.867902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 961/1915] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:33.872069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:33.881920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb07c7-9e3f-4f8e-830d-e1437b80f232 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:33.888599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:33.889486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.889773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.890385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] PATH_INFO: `/v3/projects/0eef9103073a4132a855c8c5b5963163/users/9b342f1e92f248ebbeba658d87b41830/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.905729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eef9103073a4132a855c8c5b5963163, user_id=9b342f1e92f248ebbeba658d87b41830, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:33.908398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:33.922324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 955/1916] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:33 2022] PUT /identity/v3/projects/0eef9103073a4132a855c8c5b5963163/users/9b342f1e92f248ebbeba658d87b41830/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:33.933079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:33.933372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:33.933680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:33.961841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] MFA Rules not processed for user `9b342f1e92f248ebbeba658d87b41830`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:33.963170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:34.033713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 962/1917] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:34.045855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:34.047559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:34.110981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e109b9-02f0-4ae7-bc8c-9c548a27f3e7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:34.115259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:34.116011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:34.116292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:34.116572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:34.125766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:34.127377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a47e15-9a4c-491a-977a-5b9f54ff3625 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:34.138525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 956/1918] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:39.889041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:39.953787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3e34c-df29-4241-a1bc-b3f0a7b1a460 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:39.959225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:39.959979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:39.960275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:39.960561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] PATH_INFO: `/v3/users/9b342f1e92f248ebbeba658d87b41830` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:39.961883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b342f1e92f248ebbeba658d87b41830)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:39.963953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:39.985887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:40.020891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b342f1e92f248ebbeba658d87b41830'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.021705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b342f1e92f248ebbeba658d87b41830'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.028676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b342f1e92f248ebbeba658d87b41830'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.036951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b342f1e92f248ebbeba658d87b41830'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.048950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b342f1e92f248ebbeba658d87b41830'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.055490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 963/1919] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:39 2022] DELETE /identity/v3/users/9b342f1e92f248ebbeba658d87b41830 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:40.060679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.061184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.061276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:40.080741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:40.081862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:40.093473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:40.148642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 957/1920] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:40.158825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:40.160879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:40.231622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4db998-744c-408d-a92b-260028d05647 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:40.236304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:40.237045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.237318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.237597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:40.248317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:40.249789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:40.261029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 964/1921] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:40.666573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:40.668403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:40.694122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392aaec-1137-47a1-ad35-4b3bf29d3a3e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:40.727303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:40.728400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.728739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.729094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] PATH_INFO: `/v3/projects/0eef9103073a4132a855c8c5b5963163` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:40.730531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eef9103073a4132a855c8c5b5963163)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:40.732595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:40.768965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:40.771159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:40.784906 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865daa5-538d-4d65-9c3d-1fb6926ae342 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:40.790420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:40.791172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.791456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.791752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] PATH_INFO: `/v3/users/1a9042133fda41768c32d3bf62e65ae3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:40.811794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9042133fda41768c32d3bf62e65ae3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:40.813798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:40.829985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eef9103073a4132a855c8c5b5963163', 'request_id': 'req-05c5778c-b57f-4e4c-8a66-576acc2c9a34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.845466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eef9103073a4132a855c8c5b5963163', 'request_id': 'req-05c5778c-b57f-4e4c-8a66-576acc2c9a34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.877436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eef9103073a4132a855c8c5b5963163', 'request_id': 'req-05c5778c-b57f-4e4c-8a66-576acc2c9a34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.910355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] The token cache is being invalidate because project 0eef9103073a4132a855c8c5b5963163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:40.912193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.913367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05c5778c-b57f-4e4c-8a66-576acc2c9a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.915083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 958/1922] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:40 2022] DELETE /identity/v3/projects/0eef9103073a4132a855c8c5b5963163 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:40.919927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331667-f409-42e6-ac11-460b989a545e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.920215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331667-f409-42e6-ac11-460b989a545e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.920493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331667-f409-42e6-ac11-460b989a545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:40.928357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9042133fda41768c32d3bf62e65ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.929038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9042133fda41768c32d3bf62e65ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.940191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9042133fda41768c32d3bf62e65ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.955448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9042133fda41768c32d3bf62e65ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.965121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6e331667-f409-42e6-ac11-460b989a545e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:40.968199 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6e331667-f409-42e6-ac11-460b989a545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:40.980808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e2bb183e-ecd5-4404-b553-7039cca1a35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9042133fda41768c32d3bf62e65ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:40.991879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 965/1923] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:40 2022] DELETE /identity/v3/users/1a9042133fda41768c32d3bf62e65ae3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:40.996508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:40.996903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:40.997347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.023153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:41.024418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.028626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:41.047898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 959/1924] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:41.057874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.059665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.059665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.082672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:41.085322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.106244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 966/1925] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:41.116901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.118261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.130915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:41.156847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 960/1926] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:41.161374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.170610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e5ec-30c7-4679-ac98-7f89bd9dbafc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.175217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.176397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.176729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.177001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.177705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:41.178009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.179614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.180952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49fc1f-efab-4278-915b-6633de8d8e5b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.184700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.185353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.185634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.185910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.191781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 961/1927] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:29:41.196295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.196824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.198391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.206795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35572-e97e-417c-b546-e9c181aa1136 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.209316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 967/1928] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:41.211582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.212223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.212671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.212950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.213616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.215533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.250490 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 962/1929] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:41 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:41.255209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.265518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f20da-30a7-443e-8548-cb2a6b35b787 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.328353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.328676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.328960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.329241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.329909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.331848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.355145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 968/1930] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:41 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:41.360013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.375973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7cac-89fa-4e50-b5e8-0a04152aac8a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.384972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.385647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.385928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.386609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.387603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.389289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.402887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 963/1931] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:41.404140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.415074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b6e8a-7e2a-4170-a524-fb110e0bcfb7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.420370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.421150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.421263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.421535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.424028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.424165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd225-4d06-4a6a-9e3b-9f68eae74afb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.435672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 969/1932] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:41.439730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.450789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74d42-5a42-4b63-9280-238ddcbdd5b1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.456058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.456757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.457060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.457332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] PATH_INFO: `/v3/projects/d3fd186ce4ca4030b191309f43d43baa/users/d737eac8ae0343df9ace919d75596800/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.473884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3fd186ce4ca4030b191309f43d43baa, user_id=d737eac8ae0343df9ace919d75596800, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.476821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.492684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 964/1933] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:41 2022] PUT /identity/v3/projects/d3fd186ce4ca4030b191309f43d43baa/users/d737eac8ae0343df9ace919d75596800/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:41.500012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.500837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.500837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.524796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.526601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.529469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:41.530867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.567334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:41.625967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 970/1934] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:41.637807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.640180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.642217 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5041-115e-41d4-9601-e8439bab3165 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.649813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.650806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.650969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.651215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] PATH_INFO: `/v3/projects/d2a3c1d60555442d82dcf062290a6155` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.663051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a3c1d60555442d82dcf062290a6155)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.666515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.750075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deed52-6175-4659-bef9-fc02be7c4a67 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.754859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a3c1d60555442d82dcf062290a6155', 'request_id': 'req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.764937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a3c1d60555442d82dcf062290a6155', 'request_id': 'req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.784212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a3c1d60555442d82dcf062290a6155', 'request_id': 'req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.792712 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:41.798321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] The token cache is being invalidate because project d2a3c1d60555442d82dcf062290a6155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:41.799165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.801102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.805285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 965/1935] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:41 2022] DELETE /identity/v3/projects/d2a3c1d60555442d82dcf062290a6155 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:41.807526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:41.809151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.814269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.815077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.815512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.817178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.819535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:41.880326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d0ad7-1f33-4890-92e3-bccaeb589e7c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:41.885223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:41.885866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:41.886265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:41.886410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] PATH_INFO: `/v3/users/861085a695924e578becfa35feeae78d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:41.887616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=861085a695924e578becfa35feeae78d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.889593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.903239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:41.905023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:41.922935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 971/1936] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:29:41.952543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '861085a695924e578becfa35feeae78d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.965876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '861085a695924e578becfa35feeae78d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.980105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '861085a695924e578becfa35feeae78d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.993788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '861085a695924e578becfa35feeae78d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:41.994551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1978fce6-d561-4267-83ab-408a9ba38c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '861085a695924e578becfa35feeae78d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.001283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 966/1937] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:41 2022] DELETE /identity/v3/users/861085a695924e578becfa35feeae78d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:42.185193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:42.186657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:42.200621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d6af-e4dd-402e-9f27-ce85b4a54cd1 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:42.240438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:42.272086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:42.272782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:42.273071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:42.273717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] PATH_INFO: `/v3/users/297d730bc6154e4ba3464de81811f998` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:42.274833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=297d730bc6154e4ba3464de81811f998)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:42.277115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:42.345035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297d730bc6154e4ba3464de81811f998'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.346700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297d730bc6154e4ba3464de81811f998'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.361213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297d730bc6154e4ba3464de81811f998'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.372680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297d730bc6154e4ba3464de81811f998'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.391818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297d730bc6154e4ba3464de81811f998'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:42.411092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:42.411663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:42.413653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:42.428754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 972/1938] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:42 2022] DELETE /identity/v3/users/297d730bc6154e4ba3464de81811f998 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:42.450516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:42.451807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:42.564098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 967/1939] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:42.595982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:42.597829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:42.728160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c815-351a-44c3-b2ac-728abdc4c075 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:42.732583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:42.732583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:42.732882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:42.733282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:42.745442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:42.747187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33559e-102b-43ce-9c1d-a0e20fdb0c05 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:42.759754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 973/1940] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:42.825037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:42.827343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:42.852235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a2fb-7f00-418a-bb32-17aa08b854e4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:42.873194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:42.873898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:42.875753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:42.876173 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] PATH_INFO: `/v3/projects/6243acf945244fd184006f2a0f07fefa` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:42.882462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6243acf945244fd184006f2a0f07fefa)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:42.888135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:42.929948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:42.996484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6243acf945244fd184006f2a0f07fefa', 'request_id': 'req-486629d2-2de5-4249-b150-42b0a6d3a5ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.003781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6243acf945244fd184006f2a0f07fefa', 'request_id': 'req-486629d2-2de5-4249-b150-42b0a6d3a5ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.012722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6243acf945244fd184006f2a0f07fefa', 'request_id': 'req-486629d2-2de5-4249-b150-42b0a6d3a5ab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.028346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] The token cache is being invalidate because project 6243acf945244fd184006f2a0f07fefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:43.028661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.032086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-486629d2-2de5-4249-b150-42b0a6d3a5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.032086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 968/1941] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:42 2022] DELETE /identity/v3/projects/6243acf945244fd184006f2a0f07fefa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:43.039371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.039371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.039371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.080324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:43.082417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:43.191171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:43.235955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 974/1942] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:43.243289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.244208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.244208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.278125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:43.279384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:43.289163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 969/1943] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:43.293088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.306315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcac32-146c-4c81-859b-54f650db3f41 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.311691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.313388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.313685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.314067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.314876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:43.315190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.318014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.337381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 975/1944] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:43.342334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.354122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd592f0b-d80c-428e-83b9-f7080425dac4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.365727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.366371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.366649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.366927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.367605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.369623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.419904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 970/1945] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:43 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:43.423565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.436220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c167a3c-444d-45d1-94f7-91a45b48020d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.574117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.576323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:43.613267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.613934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.614218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.614498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.615196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.618862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.672602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 976/1946] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:43 2022] POST /identity/v3/users => generated 295 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:43.700379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.726903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab3194-ea89-4f8c-a5d5-7a0c1a2251f4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.731154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd53ae-d396-420b-a6b9-c18cd9e8cd5d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.735690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.737469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.737469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.737469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.737781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.739265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.751402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.752358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.752358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.752627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] PATH_INFO: `/v3/projects/820dc044083246338c9c70b890ffd920` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.755259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820dc044083246338c9c70b890ffd920)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.758061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.767604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 977/1947] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:43.768988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.790957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af44ddd-f23e-493b-b272-481dd3dc876c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.800644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:43.802336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.802672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.803004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.804080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:43.810690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:43.837985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 978/1948] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:43.840433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:43.845813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820dc044083246338c9c70b890ffd920', 'request_id': 'req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.854165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820dc044083246338c9c70b890ffd920', 'request_id': 'req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.864427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da136-4ecf-4e31-b2b0-34fe30412439 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:43.865479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820dc044083246338c9c70b890ffd920', 'request_id': 'req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.896245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] The token cache is being invalidate because project 820dc044083246338c9c70b890ffd920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:43.896610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.897864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2fda942-f046-484b-8b0e-57f4f2a4a0c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:43.902883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 971/1949] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:43 2022] DELETE /identity/v3/projects/820dc044083246338c9c70b890ffd920 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:43.909698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dda6-91b5-419e-815d-d8e3fafb4275 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:43.909984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dda6-91b5-419e-815d-d8e3fafb4275 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:43.910316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8dda6-91b5-419e-815d-d8e3fafb4275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:43.944899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:43.981641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0ce8dda6-91b5-419e-815d-d8e3fafb4275 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:43.985384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0ce8dda6-91b5-419e-815d-d8e3fafb4275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:44.042785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.043481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.043764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.044087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] PATH_INFO: `/v3/projects/e99d20bc8dfb4e76af8fcd3350503248/users/2dd9796300ce49e1bf628478a4e3c2e5/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.090647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99d20bc8dfb4e76af8fcd3350503248, user_id=2dd9796300ce49e1bf628478a4e3c2e5, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.098588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5ad9b-5133-4315-9ea2-560f9bb16843 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.141198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 979/1950] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:43 2022] PUT /identity/v3/projects/e99d20bc8dfb4e76af8fcd3350503248/users/2dd9796300ce49e1bf628478a4e3c2e5/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:44.143402 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.143703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.144541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.155404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 972/1951] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:44.159164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.159471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.159777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.186808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:44.189485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:44.195459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] MFA Rules not processed for user `2dd9796300ce49e1bf628478a4e3c2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:44.196214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:44.254347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:44.315429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 973/1952] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:44.320224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.321178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 980/1953] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:44.325366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.327472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:44.341313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea482a9-a703-4d70-b615-38d3f1807b9b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.343601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf5610-3c2c-493d-8035-d1876032d099 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.352329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.352980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.353265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.353545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.354516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:44.354628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.357067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.359106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.359780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.360082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.360354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.361035 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.362978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.388466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 974/1954] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:44.390045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.402802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbad401-47b0-4d76-aaef-61c4afc1143a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.414448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.415253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.415623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.416108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.416849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.418590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 981/1955] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:44.419250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.419474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.477765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 975/1956] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:44.480324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.571870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10708c-97d1-4d35-80bb-29cd6feff946 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.631163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21389f-9866-4897-b1fe-3112de9cacaf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.637536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.638182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.639066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.639421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.640212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.642703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.690658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:44.702014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 976/1957] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:44.703592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.744972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab622-86bd-4c19-870d-46bfa4349c32 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.746737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.748965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.749259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.749540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.750255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.756125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.776144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.776964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.777444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.777541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.780170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.785385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.801230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 982/1958] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/users => generated 300 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:44.803142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.829239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c1b-b0dc-4aff-8c30-a1be47f6d7a4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.835654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 977/1959] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:44.839501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.845404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.846277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.846594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.846877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.847898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.849275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.857034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a512f-3cdf-4e06-98f4-dad06d3e903f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.870731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 983/1960] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:44.873554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.874482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.874975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.875625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.876596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.877322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.879164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.895961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f79d13-02a3-467c-9a39-768c057bc079 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.898135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 978/1961] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:44.900437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:44.902718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.903970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.904441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.904902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] PATH_INFO: `/v3/projects/be34be42e1d6401fa9857c15ccdac7f0/users/b608841fc0bc48fe97b2e6bcbf672ae6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.914945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ec49-d097-4cff-a1a9-dbd2bc410efc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:44.920117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:44.920892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.921384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.921729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] PATH_INFO: `/v3/projects/b94baecf5e3447ae84837f60c7d7342b/users/6fe195d655244c40a0bdf10ca7d2a323/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:44.940351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94baecf5e3447ae84837f60c7d7342b, user_id=6fe195d655244c40a0bdf10ca7d2a323, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.943758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fcdae-5649-4f6e-87a0-b6bba720b54a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.969913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] RBAC: Authorizing `identity:create_grant(project_id=be34be42e1d6401fa9857c15ccdac7f0, user_id=b608841fc0bc48fe97b2e6bcbf672ae6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:44.975846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:44.986396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 979/1962] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:44 2022] PUT /identity/v3/projects/b94baecf5e3447ae84837f60c7d7342b/users/6fe195d655244c40a0bdf10ca7d2a323/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:44.997520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:44.997816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:44.998119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.018619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 984/1963] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:44 2022] PUT /identity/v3/projects/be34be42e1d6401fa9857c15ccdac7f0/users/b608841fc0bc48fe97b2e6bcbf672ae6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:45.023506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.053338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:45.079430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] MFA Rules not processed for user `6fe195d655244c40a0bdf10ca7d2a323`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:45.080739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:45.199072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ba0f9-bde7-4175-8c2a-107d830a58be admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.209288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.210768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.211275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.211750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.212664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.214343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.235889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 985/1964] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:45.239998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.256435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28442ae-a0ea-4f0a-90aa-342f8198351b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.266099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.267082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.267461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.267744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] PATH_INFO: `/v3/projects/be34be42e1d6401fa9857c15ccdac7f0/users/b608841fc0bc48fe97b2e6bcbf672ae6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.272272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be34be42e1d6401fa9857c15ccdac7f0, user_id=b608841fc0bc48fe97b2e6bcbf672ae6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.275940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 980/1965] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:45.280081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.282619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3242da5-6178-4d7d-b5e1-a8b45ac2c1b1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.294576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0e05-0ecd-4f3d-9d22-1f824862c18a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.308685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 986/1966] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:45 2022] PUT /identity/v3/projects/be34be42e1d6401fa9857c15ccdac7f0/users/b608841fc0bc48fe97b2e6bcbf672ae6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:45.310618 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.310983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.311332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.355553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] MFA Rules not processed for user `b608841fc0bc48fe97b2e6bcbf672ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:45.357097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:45.402617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.403264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.403552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.403854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.404545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.411108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.451954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 981/1967] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:45.454756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.468582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 987/1968] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:45.479062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.480496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:45.505410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:45.553671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e4d0-9bbf-47e8-8afe-5cd3efc2fd53 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.558814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.559424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.559703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.560176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.560855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.563015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.577672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068f28e-c0fb-4848-bab1-5cddf9b00fa6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.590989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 982/1969] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:45 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:45.595052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.606062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5767-a59c-4d2a-a187-b38e6a7cd433 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.616875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.617516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.617803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.618085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.618799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.621034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.623776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:45.634936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 983/1970] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:45.669430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.672548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.674787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.675480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.675770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.704064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369bdab-8fcb-45fc-8942-4ecae80fc6fe admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.717970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.718910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.720272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.720627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.722744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.727100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.743082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 984/1971] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:45.747256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:45.761537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6bccc-9850-47ed-8f5a-60264434c8c6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:45.779480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:45.779480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.779480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.779480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] PATH_INFO: `/v3/projects/604323b05faf47d3a64938665a498782/users/024b8f2cde1f4d4faeddb643d82af5f5/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.821275 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.821498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604323b05faf47d3a64938665a498782, user_id=024b8f2cde1f4d4faeddb643d82af5f5, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:45.822851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27b511-fdb8-424a-99b1-bbde8bec3890 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.824726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:45.850928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 985/1972] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:45 2022] PUT /identity/v3/projects/604323b05faf47d3a64938665a498782/users/024b8f2cde1f4d4faeddb643d82af5f5/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:45.853092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 988/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:29:45.862662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:45.863417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:45.864096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:45.933695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] MFA Rules not processed for user `024b8f2cde1f4d4faeddb643d82af5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:45.934970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:46.111437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 986/1974] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:46.116205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.130164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11594adb-6884-4f91-b4a3-f93b60bf2659 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:46.267522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:46.269286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.269719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.270155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.271011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:46.274263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:46.399958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.414916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 989/1975] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:46 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:46.417637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1abc6-6097-4dae-93de-02f0355a1fb9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:46.450561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.591243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:46.603311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.603311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.603311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.603311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:46.603311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:46.603805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc63b2e-3f40-4fe8-8cbb-0b3b5330f971 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:46.619426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:46.623933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.625030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.625690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] PATH_INFO: `/v3/users/2dd9796300ce49e1bf628478a4e3c2e5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.627335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:46.627705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd9796300ce49e1bf628478a4e3c2e5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:46.632890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:46.656225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:46.658017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 987/1976] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:46 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:46.666332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.690967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edcc01-a6e8-43fa-9132-ed903243d2b1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:46.717084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:46.719680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.722914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.723289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.725623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:46.730132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:46.759345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 988/1977] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:46.760341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.766579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd9796300ce49e1bf628478a4e3c2e5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:46.767590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd9796300ce49e1bf628478a4e3c2e5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:46.782011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd9796300ce49e1bf628478a4e3c2e5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:46.787035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8989d7-c622-4785-b2fd-36d2074f6a93 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:46.801824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:46.805729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.807324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd9796300ce49e1bf628478a4e3c2e5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:46.807524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.807939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] PATH_INFO: `/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959/users/9e9673d5601b402697585ff34f0e27e0/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.843065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f30ac63f-8ca3-4de5-96ff-4c4efe10b14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd9796300ce49e1bf628478a4e3c2e5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:46.853334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d5dfb2cfa64d5bb76b2a266ecce959, user_id=9e9673d5601b402697585ff34f0e27e0, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:46.858121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:46.860291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 990/1978] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:46 2022] DELETE /identity/v3/users/2dd9796300ce49e1bf628478a4e3c2e5 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:46.865107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:46.865516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:46.865914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:46.888926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:46.890165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:46.891259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 989/1979] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:46 2022] PUT /identity/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959/users/9e9673d5601b402697585ff34f0e27e0/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:46.895440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:46.901962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:47.026759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 991/1980] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:47.029209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2467611-6dbd-4d90-9b55-ea364d06b2ea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:47.041625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:47.042395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:47.042978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:47.043197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:47.043406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:47.044300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:47.046553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:47.046973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:47.066377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 990/1981] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:47.068054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:47.099531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1f8b-0698-4d02-81ec-4a2dbb557895 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:47.120684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:47.121484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:47.126746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:47.127207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] PATH_INFO: `/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959/users/9e9673d5601b402697585ff34f0e27e0/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:47.144702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d5dfb2cfa64d5bb76b2a266ecce959, user_id=9e9673d5601b402697585ff34f0e27e0, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:47.155494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d91ad1-75bf-4c6b-8d61-3a05844290b8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:47.198827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 991/1982] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:47 2022] PUT /identity/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959/users/9e9673d5601b402697585ff34f0e27e0/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:47.211392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:47.211759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:47.212132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:47.219657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7e71c-5db7-4db0-ad32-7365f72a3b03 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:47.288808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] MFA Rules not processed for user `9e9673d5601b402697585ff34f0e27e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:47.295256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:47.321550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:47.350233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:47.351017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:47.351310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:47.351592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:47.553036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:47.554565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:47.575981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 992/1983] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:47.595636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 992/1984] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:47.607807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:47.610104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:47.709664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce126ef7-9d85-4757-a8b3-ef89c0b15301 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:47.729406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:47.730140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:47.730490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:47.730841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:47.732794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:47.857491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:47.859727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:47.895008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 993/1985] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:48.527160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:48.528811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:48.556951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7dfa-520c-4736-b9f9-d3e089394352 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:48.584630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:48.585316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:48.585601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:48.585875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] PATH_INFO: `/v3/projects/e99d20bc8dfb4e76af8fcd3350503248` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:48.587894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99d20bc8dfb4e76af8fcd3350503248)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:48.589881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:48.663810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:48.737146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99d20bc8dfb4e76af8fcd3350503248', 'request_id': 'req-a31919f6-a1c3-4191-b4f5-807b43100a8a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:48.750221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99d20bc8dfb4e76af8fcd3350503248', 'request_id': 'req-a31919f6-a1c3-4191-b4f5-807b43100a8a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:48.779121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99d20bc8dfb4e76af8fcd3350503248', 'request_id': 'req-a31919f6-a1c3-4191-b4f5-807b43100a8a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:48.805376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] The token cache is being invalidate because project e99d20bc8dfb4e76af8fcd3350503248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:48.805590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:48.806274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:48.808302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 993/1986] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:48 2022] DELETE /identity/v3/projects/e99d20bc8dfb4e76af8fcd3350503248 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:48.812012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:48.833676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd673-80ef-4727-bfae-74ab38134e84 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:48.977430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:49.008648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:49.009334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:49.009619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:49.009894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] PATH_INFO: `/v3/users/b608841fc0bc48fe97b2e6bcbf672ae6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:49.011348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b608841fc0bc48fe97b2e6bcbf672ae6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:49.013581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:49.117717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b608841fc0bc48fe97b2e6bcbf672ae6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:49.119149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b608841fc0bc48fe97b2e6bcbf672ae6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:49.136455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b608841fc0bc48fe97b2e6bcbf672ae6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:49.149679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b608841fc0bc48fe97b2e6bcbf672ae6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:49.198045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4f13c9a-2518-4cef-85d8-ca57f5c2ee6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b608841fc0bc48fe97b2e6bcbf672ae6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:49.213217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 994/1987] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:48 2022] DELETE /identity/v3/users/b608841fc0bc48fe97b2e6bcbf672ae6 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:49.777638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:49.779947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:49.932976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:49.933475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:49.933913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:49.984827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31919f6-a1c3-4191-b4f5-807b43100a8a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.015963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] MFA Rules not processed for user `d737eac8ae0343df9ace919d75596800`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:50.017899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.156184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.156885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.157452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.158682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.162636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.201957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 995/1988] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:50.202664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.206803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.284851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.288047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.311211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 994/1989] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:50.313729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.325240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.353936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:50.377736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53ca1c-32e6-4576-98c4-2e165d152193 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.390097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:50.391065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.392599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.392892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.393183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] PATH_INFO: `/v3/projects/be34be42e1d6401fa9857c15ccdac7f0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.396901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be34be42e1d6401fa9857c15ccdac7f0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.398870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.489956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd93c-31ea-459d-a4b9-b41537a5a270 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.505876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.509389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.509680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.510962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.512832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.524986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.558469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be34be42e1d6401fa9857c15ccdac7f0', 'request_id': 'req-2b453291-7aaa-4a65-b110-aa804dcb2299'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:50.576501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be34be42e1d6401fa9857c15ccdac7f0', 'request_id': 'req-2b453291-7aaa-4a65-b110-aa804dcb2299'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:50.585129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 995/1990] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:50 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:50.589351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.591913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be34be42e1d6401fa9857c15ccdac7f0', 'request_id': 'req-2b453291-7aaa-4a65-b110-aa804dcb2299'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:50.625326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] The token cache is being invalidate because project be34be42e1d6401fa9857c15ccdac7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:29:50.625612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:50.627609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2b453291-7aaa-4a65-b110-aa804dcb2299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:50.633403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 996/1991] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:29:50 2022] DELETE /identity/v3/projects/be34be42e1d6401fa9857c15ccdac7f0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:50.635053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649025-1b8d-44d3-8d48-39ecaa673339 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.635345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649025-1b8d-44d3-8d48-39ecaa673339 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.635621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649025-1b8d-44d3-8d48-39ecaa673339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.689680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b7649025-1b8d-44d3-8d48-39ecaa673339 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:50.690974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b7649025-1b8d-44d3-8d48-39ecaa673339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.714126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d622c03-911d-4c7f-b547-ca138e4ad282 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.721713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.723303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.723800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.724610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.725608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.727950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.766770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 996/1992] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:50 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:50.772143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.778428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.815287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15476-60d0-429d-ab42-2ca99f8c0a93 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.826722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.828374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.828668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.828950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.829886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.834183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.849858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 997/1993] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:50.859448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.859741 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.860034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.868316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 997/1994] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:50.871063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.897012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0a283-a88d-416d-a3b8-2f06bc797eb7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.911669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:50.916211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:50.934298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.934298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.934298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.934298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] PATH_INFO: `/v3/projects/bf5c5a38ffe540ebada04f8311310cd8/users/7ddfadcef85f49d6a4156ee1dccba56c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.949120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 998/1995] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:50.951743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:50.968727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5c5a38ffe540ebada04f8311310cd8, user_id=7ddfadcef85f49d6a4156ee1dccba56c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.969943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7309-51b8-4bf0-ae29-6bf3e530bdf0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:50.971476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:50.990284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:50.990974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:50.991260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:50.992010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:50.992825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:29:50.993162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:50.994985 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.025978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 998/1996] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:50 2022] PUT /identity/v3/projects/bf5c5a38ffe540ebada04f8311310cd8/users/7ddfadcef85f49d6a4156ee1dccba56c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:51.030257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.037240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 999/1997] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:29:51.038331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.063314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210da0e-0891-4422-a637-07dc8bbf0d6c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.190173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.192409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.192700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.193062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.194074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.201446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.226150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:51.244065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:51.260323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928c191-b982-4141-9d49-294eb175680e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.277062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.278307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.278602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.278890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.279613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.286256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.287548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1000/1998] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:51 2022] POST /identity/v3/projects => generated 409 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:51.289389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.325462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 999/1999] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:51.345110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.363263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fef398-92a5-431c-9470-da36c20adb59 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.447803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b22f8-486f-4db1-b5e0-b60ac3d725e4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.453238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.453962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.454579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.456751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.457608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.459820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.479318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.480247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.480747 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.481644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] PATH_INFO: `/v3/projects/bf5c5a38ffe540ebada04f8311310cd8/users/7ddfadcef85f49d6a4156ee1dccba56c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.484251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5c5a38ffe540ebada04f8311310cd8, user_id=7ddfadcef85f49d6a4156ee1dccba56c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.488294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.496206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1001/2000] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:51 2022] POST /identity/v3/users => generated 305 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:51.497197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.500241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:51.510320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1000/2001] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:51 2022] PUT /identity/v3/projects/bf5c5a38ffe540ebada04f8311310cd8/users/7ddfadcef85f49d6a4156ee1dccba56c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:51.511259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.603344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5d82-5506-463a-a62b-67364a6a11dd admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.653381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257adf5-dc04-4887-aaa7-24fb7aee7dba admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.666729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.667377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.667663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.668029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.668750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.670463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.682518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1001/2002] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:51.684020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.685367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.685652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.685924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.686646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.687729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:51.702316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5690bd-4575-41af-8b3c-df22515252ef admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.708303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.709150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.709620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.710232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.711010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.715663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.730243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1002/2003] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:51.734366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:51.746161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:51.747792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa615b-d670-4d0b-ac83-ee8f7c5b244b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:51.757391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:51.758312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.758619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.759031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] PATH_INFO: `/v3/projects/127a94bf75694277b3b8d58febdf5b3f/users/fce8570cdb874113a70191beb1f5fdd4/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.773558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.775226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f2db3-100e-447a-8b0f-0a633307ba32 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.785068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] RBAC: Authorizing `identity:create_grant(project_id=127a94bf75694277b3b8d58febdf5b3f, user_id=fce8570cdb874113a70191beb1f5fdd4, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:51.788620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:51.793647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1002/2004] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:51.817901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1003/2005] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:51 2022] PUT /identity/v3/projects/127a94bf75694277b3b8d58febdf5b3f/users/fce8570cdb874113a70191beb1f5fdd4/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:51.830178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:51.830486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:51.830745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:51.873588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] MFA Rules not processed for user `fce8570cdb874113a70191beb1f5fdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:51.874823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:51.975631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1003/2006] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:51.980787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.068799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:52.084383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44698476-eb29-424d-90a0-86d9c543467c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.089924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.090585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.090902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.091257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.091963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.094012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.130046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1004/2007] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:29:51 2022] POST /identity/v3/projects => generated 407 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:29:52.131989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.146786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db98bd-ce14-4820-aa93-f543819cf662 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.240542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.241116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.241407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.242008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.242706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.244788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.279257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1004/2008] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:29:52 2022] POST /identity/v3/users => generated 310 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:29:52.284347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.297249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3104d-b141-4619-83a0-7b9d422cdaa4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.304409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.305558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.306078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.306540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.307584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.309633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.322672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1005/2009] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:29:52.327014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.338717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30cd3-6107-4edc-9b14-790ca32983b8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.343623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.344506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.344846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.345500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] PATH_INFO: `/v3/projects/8e29509b2d87465891824adbf7c5c73d/users/d4d3d8705b5f4c91acd698b23607ba8a/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.369306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e29509b2d87465891824adbf7c5c73d, user_id=d4d3d8705b5f4c91acd698b23607ba8a, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.372581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.389429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1005/2010] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:52 2022] PUT /identity/v3/projects/8e29509b2d87465891824adbf7c5c73d/users/d4d3d8705b5f4c91acd698b23607ba8a/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:52.393367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.404211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d713f-7b8b-41ed-b6c3-ec4ed61a6f9c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.477891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.478565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.478816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.479094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.479943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.481476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee6214-9102-4ea6-8a05-080a74539b64 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.494409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1006/2011] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:29:52.498911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.512386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4805d-5ccb-49da-8807-637fe0f1aac7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.519312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.520480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.521639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.521998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] PATH_INFO: `/v3/projects/8e29509b2d87465891824adbf7c5c73d/users/d4d3d8705b5f4c91acd698b23607ba8a/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.524229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e29509b2d87465891824adbf7c5c73d, user_id=d4d3d8705b5f4c91acd698b23607ba8a, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.527581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.541894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1006/2012] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:29:52 2022] PUT /identity/v3/projects/8e29509b2d87465891824adbf7c5c73d/users/d4d3d8705b5f4c91acd698b23607ba8a/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:29:52.552187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.553046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.553542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.602628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] MFA Rules not processed for user `d4d3d8705b5f4c91acd698b23607ba8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:52.603293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:52.714275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1007/2013] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:29:52.721723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:52.723468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:52.764632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:52.797086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fd715-0245-487a-87d5-5b732808c74b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:52.807797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:52.808705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:52.809071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:52.809825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:52.822045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:52.823500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9a6ca-013f-4f43-a436-fa05470918a1 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:52.835769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1007/2014] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:53.165289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:53.180733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac251c0a-8b0a-4ff0-a4b3-5d38edc6088c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:53.184672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:53.185461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:53.185740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:53.186015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:53.282028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:53.304952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:53.317342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:53.342438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1008/2015] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:29:57.879269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:57.879795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:57.879795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:57.918815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] MFA Rules not processed for user `7ddfadcef85f49d6a4156ee1dccba56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:57.920564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:58.014751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1008/2016] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:29:58.028765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:58.030295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:58.117484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfec60-f3e6-42c8-8b1d-4861ef4ba237 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:58.122928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:58.123563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:58.123860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:58.124131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:58.125912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:58.196960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:58.198652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492daf4-8dcc-4e87-a532-9c0a45430049 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:58.212873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1009/2017] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:29:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:29:59.651817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:29:59.654012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:29:59.746683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7465658-42de-4dcb-8d89-f36a20140726 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:29:59.752346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:29:59.753137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:59.753429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:59.753713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] PATH_INFO: `/v3/users/a1b78f227cfe432f948473f915a8b6ab` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:59.754922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b78f227cfe432f948473f915a8b6ab)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:29:59.757098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:29:59.830408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b78f227cfe432f948473f915a8b6ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:59.831728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b78f227cfe432f948473f915a8b6ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:59.845875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b78f227cfe432f948473f915a8b6ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:59.856562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:29:59.860736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b78f227cfe432f948473f915a8b6ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:59.881735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b78f227cfe432f948473f915a8b6ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:29:59.888537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1009/2018] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:29:59 2022] DELETE /identity/v3/users/a1b78f227cfe432f948473f915a8b6ab => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:29:59.893964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:29:59.894479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:29:59.895010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:29:59.922457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:29:59.925169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.054205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:00.055665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.112906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1010/2019] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:00.125942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:00.130872 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.159614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:00.210056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9799-afe1-4e74-bdf6-1afa4f653c39 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:00.218731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:00.220316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:00.220954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:00.222384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] PATH_INFO: `/v3/users/6fe195d655244c40a0bdf10ca7d2a323` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:00.224572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe195d655244c40a0bdf10ca7d2a323)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:00.228834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:00.230107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2409cde-d773-4bd1-a805-43e10ed022c3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:00.235601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:00.236315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:00.236696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:00.237123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:00.254822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:00.255959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123075d-b3bb-447c-8d00-1f88bc4ce7cb service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:00.271247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1011/2020] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:30:00.346301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe195d655244c40a0bdf10ca7d2a323'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.347487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe195d655244c40a0bdf10ca7d2a323'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.367565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe195d655244c40a0bdf10ca7d2a323'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.427306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe195d655244c40a0bdf10ca7d2a323'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.460883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe195d655244c40a0bdf10ca7d2a323'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.474032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1010/2021] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:00 2022] DELETE /identity/v3/users/6fe195d655244c40a0bdf10ca7d2a323 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:00.475719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:00.476073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:00.476455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:00.513870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:00.515929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.688280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1012/2022] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:00.696599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:00.699745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.808814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca98b5f-b2cb-42d6-bc19-a3a47dc9f92b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:00.835769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:00.836648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:00.837101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:00.837526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:00.855850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:00.859343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:00.865337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:00.866806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c35b94-b169-48ab-9713-905b874f26a7 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:00.876841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:00.883152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be07e-03b4-41b0-8814-9cb657ecd918 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:00.890297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:00.891639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:00.891639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:00.891902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] PATH_INFO: `/v3/projects/ce154589d8bd48e8ab445ad28c7dbfdd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:00.893611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce154589d8bd48e8ab445ad28c7dbfdd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:00.896378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:00.899033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1011/2023] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:00.981389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce154589d8bd48e8ab445ad28c7dbfdd', 'request_id': 'req-62fa4a0c-276c-4a22-a2a3-0ae637351993'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:00.990575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce154589d8bd48e8ab445ad28c7dbfdd', 'request_id': 'req-62fa4a0c-276c-4a22-a2a3-0ae637351993'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.000410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce154589d8bd48e8ab445ad28c7dbfdd', 'request_id': 'req-62fa4a0c-276c-4a22-a2a3-0ae637351993'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.019741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] The token cache is being invalidate because project ce154589d8bd48e8ab445ad28c7dbfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:01.020060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.020840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-62fa4a0c-276c-4a22-a2a3-0ae637351993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.027351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1013/2024] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:00 2022] DELETE /identity/v3/projects/ce154589d8bd48e8ab445ad28c7dbfdd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:01.028726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.029073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.029303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.053126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:01.054441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.133070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1012/2025] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:01.142593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.142899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.143183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.175472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:01.177507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.195000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1014/2026] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:01.196504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.227153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b088-1d6b-4e38-8503-3fd5b9fbf7fd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.236029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.237066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.238272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.238503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.239233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:01.239628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.247572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.263981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1013/2027] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:30:01.267811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.282670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eda1f-8a6b-4e8f-81fa-ac6e35745872 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.302994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.303626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.304163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.304441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.306457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.310494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.351051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1015/2028] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:01 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:01.355391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.367254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.369625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22e53-7e4d-4cda-acf5-7db3d22784fc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.371214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.408313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:01.444267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.444970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.445381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.445997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.446746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.449993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.473437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587323-fa67-4f52-8358-c3f58e0c99c9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.482310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1014/2029] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:01 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:01.483928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.484256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.485311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.486119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.486796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] PATH_INFO: `/v3/projects/b94baecf5e3447ae84837f60c7d7342b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.489253 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b94baecf5e3447ae84837f60c7d7342b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.491880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.500200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88488411-ea79-4777-83e6-11205e9fd105 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.510778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.511423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.511708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.519744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.520688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.524435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.560038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1015/2030] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:01.572388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.590084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c68a39-e192-4d51-ad58-aaeae5d32f3c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.637076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b94baecf5e3447ae84837f60c7d7342b', 'request_id': 'req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.649155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b94baecf5e3447ae84837f60c7d7342b', 'request_id': 'req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.665784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b94baecf5e3447ae84837f60c7d7342b', 'request_id': 'req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.686014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] The token cache is being invalidate because project b94baecf5e3447ae84837f60c7d7342b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:01.686462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.687302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.689245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1016/2031] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:01 2022] DELETE /identity/v3/projects/b94baecf5e3447ae84837f60c7d7342b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:01.696297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.698311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.701974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.703197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.703718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.704228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.705179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.708616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.716033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:01.729168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1016/2032] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:01.731059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:01.734391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4377ef-11d4-4a4d-bac4-8c1b53096a05 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.738393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.754321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.755058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.755343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.755615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] PATH_INFO: `/v3/users/024b8f2cde1f4d4faeddb643d82af5f5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.758008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024b8f2cde1f4d4faeddb643d82af5f5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.759980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.765730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240ef6f-ced1-4cf6-b396-3a4cca700000 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:01.773133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:01.773983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.774370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.774753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] PATH_INFO: `/v3/projects/c506ac31f91b4a829437b82214d540b2/users/6cf5fb48fcbb46fbb134f2665ed04f6c/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.810203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c506ac31f91b4a829437b82214d540b2, user_id=6cf5fb48fcbb46fbb134f2665ed04f6c, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:01.813644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dbb75-cff9-47d2-ada4-03d5b7e53f7f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:01.852086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024b8f2cde1f4d4faeddb643d82af5f5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.856074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1017/2033] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:01 2022] PUT /identity/v3/projects/c506ac31f91b4a829437b82214d540b2/users/6cf5fb48fcbb46fbb134f2665ed04f6c/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:01.864902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109179-3956-4841-b36f-c13c97562c55 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:01.865261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109179-3956-4841-b36f-c13c97562c55 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:01.866425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109179-3956-4841-b36f-c13c97562c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:01.867238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024b8f2cde1f4d4faeddb643d82af5f5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.888433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024b8f2cde1f4d4faeddb643d82af5f5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.898255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024b8f2cde1f4d4faeddb643d82af5f5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.899006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024b8f2cde1f4d4faeddb643d82af5f5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:01.903859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-84109179-3956-4841-b36f-c13c97562c55 None None] MFA Rules not processed for user `6cf5fb48fcbb46fbb134f2665ed04f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:01.905117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-84109179-3956-4841-b36f-c13c97562c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:01.906589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1017/2034] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:01 2022] DELETE /identity/v3/users/024b8f2cde1f4d4faeddb643d82af5f5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:02.023690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1018/2035] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:02.039742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:02.042331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:02.092134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:02.165039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93077b-5c51-4ea1-b712-3c05f741c3d9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:02.177942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:02.178906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:02.179639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:02.180098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:02.200476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:02.202924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:02.230351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1018/2036] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:02.648049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84109179-3956-4841-b36f-c13c97562c55 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:02.649761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-84109179-3956-4841-b36f-c13c97562c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:02.754308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:02.755788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:02.837123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84109179-3956-4841-b36f-c13c97562c55 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:02.851814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:02.853093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:02.853707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:02.854235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] PATH_INFO: `/v3/users/fce8570cdb874113a70191beb1f5fdd4` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:02.856442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce8570cdb874113a70191beb1f5fdd4)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:02.858559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:02.888557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d51b60-765b-4c8e-aab7-02d9adf3e3a0 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:02.893474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:02.894146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:02.894427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:02.894703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] PATH_INFO: `/v3/projects/604323b05faf47d3a64938665a498782` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:02.897370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604323b05faf47d3a64938665a498782)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:02.900699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:02.949114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce8570cdb874113a70191beb1f5fdd4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:02.949936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce8570cdb874113a70191beb1f5fdd4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:02.973005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce8570cdb874113a70191beb1f5fdd4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:02.983886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce8570cdb874113a70191beb1f5fdd4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:02.991110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:02.995738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604323b05faf47d3a64938665a498782', 'request_id': 'req-e6c32535-99d9-4415-9293-ad12b0228928'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.005783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604323b05faf47d3a64938665a498782', 'request_id': 'req-e6c32535-99d9-4415-9293-ad12b0228928'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.009169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17ead6f9-fb61-49ea-af3c-c36fb0026d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce8570cdb874113a70191beb1f5fdd4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.017359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1019/2037] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:02 2022] DELETE /identity/v3/users/fce8570cdb874113a70191beb1f5fdd4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:03.022407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604323b05faf47d3a64938665a498782', 'request_id': 'req-e6c32535-99d9-4415-9293-ad12b0228928'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.023125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:03.023415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:03.023741 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:03.042782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] The token cache is being invalidate because project 604323b05faf47d3a64938665a498782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:03.043080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.043882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.048553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1019/2038] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:02 2022] DELETE /identity/v3/projects/604323b05faf47d3a64938665a498782 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:03.048720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:03.048720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:03.049994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:03.052901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:03.125898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:03.129974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1020/2039] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:03.139425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:03.140762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c32535-99d9-4415-9293-ad12b0228928 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:03.141279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:03.147547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:03.148316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:03.148727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:03.149076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] PATH_INFO: `/v3/users/9e9673d5601b402697585ff34f0e27e0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:03.150421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9673d5601b402697585ff34f0e27e0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:03.152516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:03.213477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9673d5601b402697585ff34f0e27e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.218094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4945-2681-426a-8b01-df7a3cfca3e3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:03.223274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:03.223963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:03.224218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:03.224584 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:03.235638 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9673d5601b402697585ff34f0e27e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.238781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:03.240733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:03.252017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9673d5601b402697585ff34f0e27e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.255689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1021/2040] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:30:03.271557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9673d5601b402697585ff34f0e27e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.272365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9673d5601b402697585ff34f0e27e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.281757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1020/2041] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:03 2022] DELETE /identity/v3/users/9e9673d5601b402697585ff34f0e27e0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:03.712808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:03.714613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:03.727709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144b42-a895-4e3d-84f0-1de250f06e6f service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:03.827493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:03.828305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:03.828590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:03.828864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] PATH_INFO: `/v3/projects/127a94bf75694277b3b8d58febdf5b3f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:03.830136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127a94bf75694277b3b8d58febdf5b3f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:03.833245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:03.886660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:03.907305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:03.908877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:03.931323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd3f1e-3218-4a58-9128-2f7599ce724c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:03.954390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127a94bf75694277b3b8d58febdf5b3f', 'request_id': 'req-8303f896-757e-42d6-826d-97fbe8905b56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.978991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127a94bf75694277b3b8d58febdf5b3f', 'request_id': 'req-8303f896-757e-42d6-826d-97fbe8905b56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:03.990797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127a94bf75694277b3b8d58febdf5b3f', 'request_id': 'req-8303f896-757e-42d6-826d-97fbe8905b56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.011575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] The token cache is being invalidate because project 127a94bf75694277b3b8d58febdf5b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:04.012016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.012785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.017751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1022/2042] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:03 2022] DELETE /identity/v3/projects/127a94bf75694277b3b8d58febdf5b3f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:04.019051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.020617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:04.055064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.055997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.056533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.056917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] PATH_INFO: `/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.061567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d5dfb2cfa64d5bb76b2a266ecce959)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.067777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.116504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303f896-757e-42d6-826d-97fbe8905b56 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.128042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.130504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.130971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.131254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] PATH_INFO: `/v3/users/d4d3d8705b5f4c91acd698b23607ba8a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.133335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d3d8705b5f4c91acd698b23607ba8a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.135310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.183493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d5dfb2cfa64d5bb76b2a266ecce959', 'request_id': 'req-82d11085-d8e6-4481-9824-bb4195e7ccae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.192105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d5dfb2cfa64d5bb76b2a266ecce959', 'request_id': 'req-82d11085-d8e6-4481-9824-bb4195e7ccae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.205337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d5dfb2cfa64d5bb76b2a266ecce959', 'request_id': 'req-82d11085-d8e6-4481-9824-bb4195e7ccae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.212324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d3d8705b5f4c91acd698b23607ba8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.213252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d3d8705b5f4c91acd698b23607ba8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.225470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d3d8705b5f4c91acd698b23607ba8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.228250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] The token cache is being invalidate because project e3d5dfb2cfa64d5bb76b2a266ecce959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:04.229702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.230809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-82d11085-d8e6-4481-9824-bb4195e7ccae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.239030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1021/2043] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:03 2022] DELETE /identity/v3/projects/e3d5dfb2cfa64d5bb76b2a266ecce959 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:04.241374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.241970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.242573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.247789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d3d8705b5f4c91acd698b23607ba8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.270882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:04.290570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3e7d7b1-91a0-41d4-b4a2-33cecd4f5b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d3d8705b5f4c91acd698b23607ba8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:04.302200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:04.307098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:04.308381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1023/2044] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:04 2022] DELETE /identity/v3/users/d4d3d8705b5f4c91acd698b23607ba8a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:04.410204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1022/2045] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:04.419121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.419603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.420037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.469895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:04.471359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:04.487676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1024/2046] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:04.491335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.504244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101169bb-3131-4210-9652-ee45bc7deb49 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.510378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.511443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.511689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.512094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.513015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:04.513340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.515197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.532213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1023/2047] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:04.535395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.562843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0621a-f5ad-44ac-b377-54e37f00428d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.569217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.569995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.570346 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.570688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.572641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.573716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.657383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.661424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1025/2048] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:04 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:04.677784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e887a-70b6-4874-b319-1725c8605c42 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.781350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.782012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.782602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.783165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.783782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.789132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.806527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.808440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:04.828526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc67fbb-6494-4362-b2b3-020d23e9cba0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.838286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1024/2049] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:04 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:04.838286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.849669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8559b-d2c8-4173-8b94-5f0da511132d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.857759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.858526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.858796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.859017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.859460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.859586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.859745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.860052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] PATH_INFO: `/v3/projects/8e29509b2d87465891824adbf7c5c73d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.860731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.861876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e29509b2d87465891824adbf7c5c73d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.863284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.865743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.872029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:04.889854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1025/2050] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:04.891268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.903328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13963-753f-47c4-9c1f-45002f34fa94 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.910356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.911175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.911549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.911965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:04.912774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:04.914612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:04.936118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1026/2051] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:04.940871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:04.960443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f221c4-7f1e-4c7f-ac21-3e13e269d295 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:04.977105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:04.979349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:04.979735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:04.980204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] PATH_INFO: `/v3/projects/382fa28f646949f0a53a537dd3a072a0/users/9648c91338aa4f4db619e0e371464e7a/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.000664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=382fa28f646949f0a53a537dd3a072a0, user_id=9648c91338aa4f4db619e0e371464e7a, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.004955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81047-a04b-4081-ae5d-6e7c27a5bad8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.008340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e29509b2d87465891824adbf7c5c73d', 'request_id': 'req-5f1fc4cf-4341-4f0b-8307-804f6027ba90'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:05.020147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e29509b2d87465891824adbf7c5c73d', 'request_id': 'req-5f1fc4cf-4341-4f0b-8307-804f6027ba90'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:05.026982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e29509b2d87465891824adbf7c5c73d', 'request_id': 'req-5f1fc4cf-4341-4f0b-8307-804f6027ba90'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:05.031474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1027/2052] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:04 2022] PUT /identity/v3/projects/382fa28f646949f0a53a537dd3a072a0/users/9648c91338aa4f4db619e0e371464e7a/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:05.043175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e25d0-0b0e-456b-b153-6b9f7153a630 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.043467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e25d0-0b0e-456b-b153-6b9f7153a630 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.043749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e25d0-0b0e-456b-b153-6b9f7153a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.051190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] The token cache is being invalidate because project 8e29509b2d87465891824adbf7c5c73d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:05.051740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:05.053124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5f1fc4cf-4341-4f0b-8307-804f6027ba90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:05.055309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1026/2053] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:04 2022] DELETE /identity/v3/projects/8e29509b2d87465891824adbf7c5c73d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:05.063196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.063545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.064398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.077272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-862e25d0-0b0e-456b-b153-6b9f7153a630 None None] MFA Rules not processed for user `9648c91338aa4f4db619e0e371464e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:05.078508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-862e25d0-0b0e-456b-b153-6b9f7153a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:05.091802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:05.093484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:05.113037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:05.178275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1028/2054] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:05.188039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1027/2055] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:05.189748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.190588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.191089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.191443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.191555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:05.229358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:05.230707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:05.239808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1029/2056] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:05.244419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.268829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27aaeb8-467c-478a-b782-88b5b6416f32 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.280565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.281190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.281468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.281740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.283444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:05.284327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.285949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.293506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173c19-9cb0-484c-b710-070bdc6f4688 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.300234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1030/2057] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:30:05.305339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.307176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.307929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.308276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.308646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.319166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.320220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c98979-2e4b-4a2d-9009-4dfad700e89f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.320758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.325153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.325841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.326200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.326553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.327292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.329446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.331195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1028/2058] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:30:05.368268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1031/2059] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:05.368995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.386668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb288f1-d1cf-4715-a4fa-f8881ee3c222 service glance] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.434996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:05.477792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.478651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.482273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.482660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.483628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.486164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.522268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1029/2060] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:05.528520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.546257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18ba2d-dbd1-414a-97ee-55ee928909ef admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.558387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.560300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.561134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.561775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.566633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.568980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.582721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1032/2061] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:05.586767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.602089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cb1b2-3c2f-4250-9227-c571a043fb11 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.612481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.613330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.613801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.614219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.615081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.616909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391f987-9dfa-4a83-8019-509707e72b6b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.647927 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1030/2062] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:05.650050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:05.661927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162bd7f-fbc4-4fb7-b27d-2b8f0638d855 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:05.668254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:05.669070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.669471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.669781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] PATH_INFO: `/v3/projects/8f0f9cd1202c4b10a330c14ec6b58f19/users/9f954b5b79e842b6a01c9f149361a6b5/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.711348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0f9cd1202c4b10a330c14ec6b58f19, user_id=9f954b5b79e842b6a01c9f149361a6b5, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:05.714618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:05.737489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1033/2063] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:05 2022] PUT /identity/v3/projects/8f0f9cd1202c4b10a330c14ec6b58f19/users/9f954b5b79e842b6a01c9f149361a6b5/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:05.746194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:05.746498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:05.746828 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:05.792737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] MFA Rules not processed for user `9f954b5b79e842b6a01c9f149361a6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:05.794874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:05.886669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1031/2064] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:05.891047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.013506 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad778b2-ba88-4657-8174-4dae3207a810 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.018318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.018937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.019215 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.019593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.021764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.024543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.063940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.067807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:06.080785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1034/2065] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:05 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:06.085892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.103684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c292d2f-bf94-4fba-8d96-d19bf7e39065 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.164160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:06.206267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958c45b-7da5-4a8a-9634-1f96c63d32e8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.212930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.213796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.214220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.214672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.215590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.219407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.227039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.228474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.229523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.231049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] PATH_INFO: `/v3/users/6cf5fb48fcbb46fbb134f2665ed04f6c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.235504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf5fb48fcbb46fbb134f2665ed04f6c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.238251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.273911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1035/2066] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:06 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:06.277363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.307334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28359858-7c06-4721-9e97-4315905a4ae5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.318589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.319621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.320146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.320589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.321402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.323174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.350537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1036/2067] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:06.351692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.369001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97395ee-3512-4ba2-a705-779e7eb300d6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.375785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.376508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.376918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.377533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.378325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.381644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.384852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:06.403329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1037/2068] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:06.404530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.414287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf5fb48fcbb46fbb134f2665ed04f6c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:06.415003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf5fb48fcbb46fbb134f2665ed04f6c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:06.419811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7353c6c-8b41-40b2-a791-00cc6500d834 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.426363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.427265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.427640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.428056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] PATH_INFO: `/v3/projects/ad833e76d14e4519a288cb202c01c6ea/users/0e1b657e3f2e4ff686f16051d7081922/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.429297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf5fb48fcbb46fbb134f2665ed04f6c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:06.459092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad833e76d14e4519a288cb202c01c6ea, user_id=0e1b657e3f2e4ff686f16051d7081922, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.462396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547312d-461a-4845-9c0e-bc01025fd441 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.464364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf5fb48fcbb46fbb134f2665ed04f6c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:06.486210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1038/2069] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:06 2022] PUT /identity/v3/projects/ad833e76d14e4519a288cb202c01c6ea/users/0e1b657e3f2e4ff686f16051d7081922/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:06.496947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.497652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.498000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.515698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9511aedb-9e46-48fa-8c2e-afe66a912db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf5fb48fcbb46fbb134f2665ed04f6c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:06.537877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1032/2070] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:06 2022] DELETE /identity/v3/users/6cf5fb48fcbb46fbb134f2665ed04f6c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:06.543258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.551227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.551227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.557428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] MFA Rules not processed for user `0e1b657e3f2e4ff686f16051d7081922`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:06.558258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:06.636465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:06.637918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:06.683414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1039/2071] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:06.688256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.690511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:06.771684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1033/2072] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:06.785020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c775306-98e5-428b-b188-9a1a840b8106 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.791388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.792346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.792840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.793191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.794087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.794354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.796279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:06.796404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:06.812908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:06.856515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1040/2073] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:06 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:06.860792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:06.914356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6c75e-682e-4648-ac9a-c9e36e475440 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.962246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fecf37-ce41-4d39-b3f9-361255d9e583 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:06.974810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:06.976362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:06.978694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:06.979085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:06.980316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:06.983381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.013811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.014768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.015197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.015784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.019957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1041/2074] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:06 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:07.024397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.030804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.032661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.034687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c8e5a-5fde-4742-985e-303a80d55621 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.040790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.041255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.041540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.041820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.042537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.044864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.051236 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1034/2075] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:07.055594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1042/2076] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:07.060428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.071204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0d13-042a-47b1-8809-0639344f4b3d service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.075724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.076425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.076690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.076964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] PATH_INFO: `/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491/users/c1cf2228b02d4cf09916e6fed37758b5/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.092751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da3a34b22f48d4aa3f42b60e4e8491, user_id=c1cf2228b02d4cf09916e6fed37758b5, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.095399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1dd9c-ca24-4af4-aa0f-60f922e49e72 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.109534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:07.118176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1035/2077] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:07 2022] PUT /identity/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491/users/c1cf2228b02d4cf09916e6fed37758b5/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:07.126636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.143487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0d346-fb95-4af7-9808-446e2c6ab2f8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.169146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a3ec3-9f4b-4cbc-bb89-7ce291f4732d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.169737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a3ec3-9f4b-4cbc-bb89-7ce291f4732d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.170032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a3ec3-9f4b-4cbc-bb89-7ce291f4732d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.171992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1036/2078] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:30:07.204446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.205122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.205435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.205711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.206415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.208522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.216829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.217290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.217573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.220862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1043/2079] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:07.225637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.235715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc4424-f38e-4f5c-8e68-57ff5efd2f74 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.242230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:07.242780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.245459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.245586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.245586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.245586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] PATH_INFO: `/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491/users/c1cf2228b02d4cf09916e6fed37758b5/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.246557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da3a34b22f48d4aa3f42b60e4e8491, user_id=c1cf2228b02d4cf09916e6fed37758b5, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.249618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addf35c-9cde-4f6f-9e04-e595d2c54b37 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.264817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1044/2080] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:07 2022] PUT /identity/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491/users/c1cf2228b02d4cf09916e6fed37758b5/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:07.277780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.278157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.278908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.317381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] MFA Rules not processed for user `c1cf2228b02d4cf09916e6fed37758b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:07.320157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.332230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1037/2081] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:07.333365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.334843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.356020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f2b5c-b934-4644-86a3-ff3874c605d3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.404813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:07.448017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1045/2082] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:07.451038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.453051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.456207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.460662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.460662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.460662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] PATH_INFO: `/v3/users/1989ddc458d645af8d017987a6b93809` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.460662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1989ddc458d645af8d017987a6b93809)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.463595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.558407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1989ddc458d645af8d017987a6b93809'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.560218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1989ddc458d645af8d017987a6b93809'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.573762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1989ddc458d645af8d017987a6b93809'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.574377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be430c9-cc38-41c7-ae44-06b9ac4f34c4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.584035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.584871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.585219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.585697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.592473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1989ddc458d645af8d017987a6b93809'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.600488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.602170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.610390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1989ddc458d645af8d017987a6b93809'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.614130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1046/2083] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:07.614954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.616317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.617377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1038/2084] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:07 2022] DELETE /identity/v3/users/1989ddc458d645af8d017987a6b93809 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:07.621324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.622485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.650486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:07.722780 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80906ee-6108-4b35-8def-c51eb4c918d2 service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.731724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f0fb0-673b-419a-ba83-6d2b18ad8149 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:07.737596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.738350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.738707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.739054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] PATH_INFO: `/v3/projects/c506ac31f91b4a829437b82214d540b2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.740531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c506ac31f91b4a829437b82214d540b2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:07.742512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:07.796795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c506ac31f91b4a829437b82214d540b2', 'request_id': 'req-cc9456f2-4a06-4a0c-883d-e3930767ee5e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.805847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:07.806036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c506ac31f91b4a829437b82214d540b2', 'request_id': 'req-cc9456f2-4a06-4a0c-883d-e3930767ee5e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.806739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:07.807088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:07.807773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:07.810906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:07.815485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c506ac31f91b4a829437b82214d540b2', 'request_id': 'req-cc9456f2-4a06-4a0c-883d-e3930767ee5e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.830036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] The token cache is being invalidate because project c506ac31f91b4a829437b82214d540b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:07.830366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.831421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:07.835877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1039/2085] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:07 2022] DELETE /identity/v3/projects/c506ac31f91b4a829437b82214d540b2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:07.895579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:07.917607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:07.920330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.011673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.014898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.044397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1047/2086] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:08.096827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9456f2-4a06-4a0c-883d-e3930767ee5e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.104165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.104165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.104165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.104483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] PATH_INFO: `/v3/projects/e1c7c86e4b1c40ec8e4335a2770353f5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.106803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.110333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.113091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c7c86e4b1c40ec8e4335a2770353f5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.116671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.131459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c7da-da2f-4ff9-98a4-7d749e1e5844 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.209014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:08.212103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c7c86e4b1c40ec8e4335a2770353f5', 'request_id': 'req-13c8031b-af1f-4e06-b2cf-78cd127d38b9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:08.222579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.223274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.223484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.223752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.226069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.240534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c7c86e4b1c40ec8e4335a2770353f5', 'request_id': 'req-13c8031b-af1f-4e06-b2cf-78cd127d38b9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:08.258583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c7c86e4b1c40ec8e4335a2770353f5', 'request_id': 'req-13c8031b-af1f-4e06-b2cf-78cd127d38b9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:08.277892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] The token cache is being invalidate because project e1c7c86e4b1c40ec8e4335a2770353f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:08.278201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:08.279802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-13c8031b-af1f-4e06-b2cf-78cd127d38b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:08.282399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1040/2087] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:07 2022] DELETE /identity/v3/projects/e1c7c86e4b1c40ec8e4335a2770353f5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:08.289153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.289440 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.290102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.322620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:08.323884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.328565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.332332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981dab6-d2a2-4a43-aa9d-eb6db57b9743 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.348228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1048/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:30:08.406063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1041/2089] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:08.415133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.415432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.415734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.435329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:08.436569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.445299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1049/2090] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:08.449658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.460835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8e3d-c65f-484e-8815-f466fcaf67e5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.465919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.466662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.466892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.467093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.467803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:08.468141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.469993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.482384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1042/2091] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:08.486913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.496560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083f406-0c84-48ff-964a-f76ed510be93 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.501313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.503144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.503423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.503690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.504393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.506318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.547866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1050/2092] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:08 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:08.551393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.563222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f761c0-b53a-4840-a59b-15b94b1e0592 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.626350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.627935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.628298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.628576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.629281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.631356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.657227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1043/2093] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:08 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:08.658520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.672711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe2cb3-6cd2-46e9-ad23-d9793a68e856 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.678739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.679364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.679649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.679966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.680784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.682403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.698502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1051/2094] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:08.700399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.719021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88736d5-b1cd-4304-8ee5-e8040a235115 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.726298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.726924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.727208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.727477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.729146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.730094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03c07b-9a8f-422e-8583-554de2e7548a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.746516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1044/2095] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:08.751165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.763666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56757a01-bfc1-4d01-8e88-69886cf4195b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.768678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.769788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.770194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.770609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] PATH_INFO: `/v3/projects/b69b9aea60544dfe94ef7e2e3659a0b8/users/3f17511222d54ede809be0b560df242f/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.788286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69b9aea60544dfe94ef7e2e3659a0b8, user_id=3f17511222d54ede809be0b560df242f, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:08.790990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:08.808109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1052/2096] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:08 2022] PUT /identity/v3/projects/b69b9aea60544dfe94ef7e2e3659a0b8/users/3f17511222d54ede809be0b560df242f/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:08.819807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.820174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.820453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:08.848455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] MFA Rules not processed for user `3f17511222d54ede809be0b560df242f`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:08.849738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.879601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:08.916963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1045/2097] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:08.929232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:08.930936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:08.958406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:08.993808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4f28-cac2-41b7-acd5-9f81783b7260 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:08.997775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:08.998420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:08.998739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:08.999053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:09.008799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:09.010235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f86593-234c-4ac9-8f1d-50fd77e3cb1a service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:09.019931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1053/2098] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:11.306860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:11.308389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:11.380510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6382d-eaaf-478e-8aa2-9db2e4879eb0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:11.385323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:11.385965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:11.386245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:11.386520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] PATH_INFO: `/v3/users/9f954b5b79e842b6a01c9f149361a6b5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:11.387676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f954b5b79e842b6a01c9f149361a6b5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:11.389910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:11.436725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f954b5b79e842b6a01c9f149361a6b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:11.437370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f954b5b79e842b6a01c9f149361a6b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:11.445885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f954b5b79e842b6a01c9f149361a6b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:11.456918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f954b5b79e842b6a01c9f149361a6b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:11.469256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f954b5b79e842b6a01c9f149361a6b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:11.475577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1046/2099] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:11 2022] DELETE /identity/v3/users/9f954b5b79e842b6a01c9f149361a6b5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:11.480025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:11.480467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:11.480614 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:11.501136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:11.502346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:11.564721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1054/2100] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:11.574658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:11.576093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:11.634938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:11.637931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb33d1-9ce2-4db9-a925-4118143ea713 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:11.642627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:11.643257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:11.644405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:11.644755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:11.654390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:11.655809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:11.666026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1047/2101] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:12.051916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:12.056793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:12.077350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002fd5f-aed9-422d-8286-b89a859e2d3c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:12.084283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:12.084931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:12.085211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:12.085486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] PATH_INFO: `/v3/projects/8f0f9cd1202c4b10a330c14ec6b58f19` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:12.087286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0f9cd1202c4b10a330c14ec6b58f19)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:12.089293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:12.140811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0f9cd1202c4b10a330c14ec6b58f19', 'request_id': 'req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.147617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0f9cd1202c4b10a330c14ec6b58f19', 'request_id': 'req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.156338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0f9cd1202c4b10a330c14ec6b58f19', 'request_id': 'req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.171094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] The token cache is being invalidate because project 8f0f9cd1202c4b10a330c14ec6b58f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:12.171429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.172116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.176736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1055/2102] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:12 2022] DELETE /identity/v3/projects/8f0f9cd1202c4b10a330c14ec6b58f19 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:12.178384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:12.190222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e9d2b-7c10-491a-8fd9-b6eba33f5c12 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:12.264248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:12.265245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:12.265468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:12.265747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] PATH_INFO: `/v3/users/0e1b657e3f2e4ff686f16051d7081922` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:12.267077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1b657e3f2e4ff686f16051d7081922)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:12.269681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:12.340249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1b657e3f2e4ff686f16051d7081922'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.341251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1b657e3f2e4ff686f16051d7081922'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.352073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1b657e3f2e4ff686f16051d7081922'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.360645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1b657e3f2e4ff686f16051d7081922'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.382223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1b657e3f2e4ff686f16051d7081922'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.389319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1048/2103] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:12 2022] DELETE /identity/v3/users/0e1b657e3f2e4ff686f16051d7081922 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:12.679235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:12.680133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:12.694525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d2a0c-79fa-4c2d-9b8e-704b3fa89a3e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:12.758922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:12.773106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:12.773783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:12.774068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:12.774345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] PATH_INFO: `/v3/projects/ad833e76d14e4519a288cb202c01c6ea` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:12.775573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad833e76d14e4519a288cb202c01c6ea)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:12.777525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:12.827798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad833e76d14e4519a288cb202c01c6ea', 'request_id': 'req-05897be6-099d-4e77-bc65-e47bbc10917c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.833494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad833e76d14e4519a288cb202c01c6ea', 'request_id': 'req-05897be6-099d-4e77-bc65-e47bbc10917c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.840922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad833e76d14e4519a288cb202c01c6ea', 'request_id': 'req-05897be6-099d-4e77-bc65-e47bbc10917c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.852857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] The token cache is being invalidate because project ad833e76d14e4519a288cb202c01c6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:12.853143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.853851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:12.855366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1056/2104] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:12 2022] DELETE /identity/v3/projects/ad833e76d14e4519a288cb202c01c6ea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:12.859375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:12.860859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:12.874327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534573-7a9c-4c60-adb2-62a5f4a0d58a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:12.948687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:12.949351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:12.949634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:12.950228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] PATH_INFO: `/v3/users/c1cf2228b02d4cf09916e6fed37758b5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:12.953363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1cf2228b02d4cf09916e6fed37758b5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:12.954553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:12.971757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:13.015442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1cf2228b02d4cf09916e6fed37758b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.016493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1cf2228b02d4cf09916e6fed37758b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.027282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1cf2228b02d4cf09916e6fed37758b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.048627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1cf2228b02d4cf09916e6fed37758b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.070269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c07a79fd-588e-4689-ac35-6ef822f2e1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1cf2228b02d4cf09916e6fed37758b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.079052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1049/2105] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:12 2022] DELETE /identity/v3/users/c1cf2228b02d4cf09916e6fed37758b5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:13.396637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:13.398092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:13.414035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05897be6-099d-4e77-bc65-e47bbc10917c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:13.493942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:13.494592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.494871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.495150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] PATH_INFO: `/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.496354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2da3a34b22f48d4aa3f42b60e4e8491)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:13.498305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:13.567222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2da3a34b22f48d4aa3f42b60e4e8491', 'request_id': 'req-0a6d83c2-727a-44ca-8281-35cb6552692c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.574240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2da3a34b22f48d4aa3f42b60e4e8491', 'request_id': 'req-0a6d83c2-727a-44ca-8281-35cb6552692c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.585000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2da3a34b22f48d4aa3f42b60e4e8491', 'request_id': 'req-0a6d83c2-727a-44ca-8281-35cb6552692c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.597940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] The token cache is being invalidate because project a2da3a34b22f48d4aa3f42b60e4e8491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:13.598234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.598869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a6d83c2-727a-44ca-8281-35cb6552692c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:13.600581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1057/2106] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:13 2022] DELETE /identity/v3/projects/a2da3a34b22f48d4aa3f42b60e4e8491 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:13.606633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.607404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.607965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.630092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:13.631310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:13.729192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1050/2107] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:13.738307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.738778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.738892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.759370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:13.760604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:13.768883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1058/2108] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:13.773581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:13.792220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b874be-f754-4e0b-8e9d-138d32533195 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:13.797313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:13.798096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.798527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.798947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.799756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:13.800108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:13.801841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:13.819721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1051/2109] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:13.823378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:13.828708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:13.840537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb977c-1f81-4667-b914-1c801db1e8d1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:13.847981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:13.848592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.848892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.849130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.849819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:13.851690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:13.880231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1059/2110] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:13 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:13.884530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:13.898541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f642fd-17c9-48d6-9511-2693645b8093 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:13.978284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57285-3754-4688-9056-068e0777612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:13.978922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57285-3754-4688-9056-068e0777612b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:13.979401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57285-3754-4688-9056-068e0777612b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:13.979739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57285-3754-4688-9056-068e0777612b admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:13.980838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57285-3754-4688-9056-068e0777612b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:13.983521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57285-3754-4688-9056-068e0777612b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.013470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1052/2111] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:13 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:14.017799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.029073 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44723a-55ee-4587-a3a7-a6040ea6cc3d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.034113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.034857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.035204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.035595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.036654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.038377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.049745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1060/2112] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:14.054281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57285-3754-4688-9056-068e0777612b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.066662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57285-3754-4688-9056-068e0777612b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.072086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.072633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.072919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.073222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.073926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.075552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829785a-aeaa-430c-b56c-370aa9784280 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.080498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:14.099139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1053/2113] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:14.105296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.126261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028322e7-bd5a-496b-a947-aa413c4ba35b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.150263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.150967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.151255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.151972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] PATH_INFO: `/v3/projects/f46cc10cd14444e7ad74050aace12ead/users/eb34170bc1d643ad9a120b1fc7b5de60/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.182384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46cc10cd14444e7ad74050aace12ead, user_id=eb34170bc1d643ad9a120b1fc7b5de60, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.198782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.240010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1061/2114] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:14 2022] PUT /identity/v3/projects/f46cc10cd14444e7ad74050aace12ead/users/eb34170bc1d643ad9a120b1fc7b5de60/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:14.252794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.253096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.253948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.346478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] MFA Rules not processed for user `eb34170bc1d643ad9a120b1fc7b5de60`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:14.348269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:14.433604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1054/2115] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:14.438239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.537986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a64e1-4bf6-4967-b716-4e899df8d7d4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.544239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.545097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.545439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.545783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.546600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.548631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.579350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1062/2116] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:14 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:14.585142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.598536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6329e0-8741-4a0b-90ba-1b5076dffe85 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.660223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.660930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.661028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.661306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.662040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.664041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.687668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1055/2117] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:14 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:14.692045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.702510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160b0a7-8f25-4dc1-b7da-dac456cfe899 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.707595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.708254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.708538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.708812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.709524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.711123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.721798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1063/2118] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:14.726522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.736628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89793-79ee-4a4f-aa7d-26645777a50a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.741373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.742053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.742339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.742619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] PATH_INFO: `/v3/projects/4035b9bbec3d4ffa808a69bea6698dec/users/5ad05de2d3784bd0a7758f9b9a1f5aa6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.758917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4035b9bbec3d4ffa808a69bea6698dec, user_id=5ad05de2d3784bd0a7758f9b9a1f5aa6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.761676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.776100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1056/2119] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:14 2022] PUT /identity/v3/projects/4035b9bbec3d4ffa808a69bea6698dec/users/5ad05de2d3784bd0a7758f9b9a1f5aa6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:14.780343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.793138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f8bd-70d2-41fd-94b4-ca26532e3ae6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.854814 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.855550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.855937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.856220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.856925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.858531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628667c4-6eb5-4cf5-abb4-6d0dd6af19aa admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.868394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1064/2120] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:14.872553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:14.882751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc6b37-f266-408c-b8a0-7e882b295d6f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:14.887390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:14.888123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.888438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.888688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] PATH_INFO: `/v3/projects/4035b9bbec3d4ffa808a69bea6698dec/users/5ad05de2d3784bd0a7758f9b9a1f5aa6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.890691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4035b9bbec3d4ffa808a69bea6698dec, user_id=5ad05de2d3784bd0a7758f9b9a1f5aa6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:14.893457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:14.905241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1057/2121] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:14 2022] PUT /identity/v3/projects/4035b9bbec3d4ffa808a69bea6698dec/users/5ad05de2d3784bd0a7758f9b9a1f5aa6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:14.916937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:14.917168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:14.917448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:14.945164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] MFA Rules not processed for user `5ad05de2d3784bd0a7758f9b9a1f5aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:14.946485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:14.952920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:15.035236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1065/2122] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:15.046060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:15.047778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:15.091171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:15.107419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ae233-3833-406c-8ab0-e1694907cad8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:15.111021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:15.111628 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:15.111933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:15.112206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:15.121272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:15.122696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d97316-c0cd-4e36-97c6-522b9d3470e2 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:15.132778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1058/2123] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:15.765993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:15.832905 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a642e-848d-4d58-aa27-15eedace45ef None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:15.837392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:15.838037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:15.838322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:15.838600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] PATH_INFO: `/v3/users/eb34170bc1d643ad9a120b1fc7b5de60` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:15.839799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb34170bc1d643ad9a120b1fc7b5de60)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:15.841786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:15.886249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb34170bc1d643ad9a120b1fc7b5de60'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:15.892538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb34170bc1d643ad9a120b1fc7b5de60'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:15.904351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb34170bc1d643ad9a120b1fc7b5de60'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:15.911608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb34170bc1d643ad9a120b1fc7b5de60'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:15.912314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb34170bc1d643ad9a120b1fc7b5de60'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:15.918234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1066/2124] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:15 2022] DELETE /identity/v3/users/eb34170bc1d643ad9a120b1fc7b5de60 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:15.923650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:15.924143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:15.924417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:15.946224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:15.947385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:16.008949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1059/2125] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:16.018579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:16.020572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:16.042479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:16.079380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b351e-b961-4d01-99ec-d26eed0dda7a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:16.085500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:16.086318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:16.086417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:16.086802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:16.098279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:16.099784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:16.109552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1067/2126] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:16.385700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:16.387244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:16.404268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979918d4-8026-439d-affc-d5d10c37d3c9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:16.409310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:16.410088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:16.410374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:16.410653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] PATH_INFO: `/v3/projects/f46cc10cd14444e7ad74050aace12ead` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:16.411838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46cc10cd14444e7ad74050aace12ead)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:16.413789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:16.467209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46cc10cd14444e7ad74050aace12ead', 'request_id': 'req-3863d24d-d1fc-4fae-bb95-31136b97b0ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.474050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46cc10cd14444e7ad74050aace12ead', 'request_id': 'req-3863d24d-d1fc-4fae-bb95-31136b97b0ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.483197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46cc10cd14444e7ad74050aace12ead', 'request_id': 'req-3863d24d-d1fc-4fae-bb95-31136b97b0ab'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.495140 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] The token cache is being invalidate because project f46cc10cd14444e7ad74050aace12ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:16.495331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.496130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.497710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1060/2127] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:16 2022] DELETE /identity/v3/projects/f46cc10cd14444e7ad74050aace12ead => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:16.501790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:16.513799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cdccf-af81-4012-8860-0cd98da38201 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:16.576791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:16.577444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:16.577723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:16.577999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] PATH_INFO: `/v3/users/5ad05de2d3784bd0a7758f9b9a1f5aa6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:16.579401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad05de2d3784bd0a7758f9b9a1f5aa6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:16.581634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:16.641166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad05de2d3784bd0a7758f9b9a1f5aa6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.653818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad05de2d3784bd0a7758f9b9a1f5aa6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.667056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad05de2d3784bd0a7758f9b9a1f5aa6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.673895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad05de2d3784bd0a7758f9b9a1f5aa6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.674580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-97bcc343-bf91-44c1-bc42-c74b4ffde621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad05de2d3784bd0a7758f9b9a1f5aa6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:16.681528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1068/2128] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:16 2022] DELETE /identity/v3/users/5ad05de2d3784bd0a7758f9b9a1f5aa6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:17.050642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.052212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.065095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d24d-d1fc-4fae-bb95-31136b97b0ab admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.122498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.123149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.123435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.123712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] PATH_INFO: `/v3/projects/4035b9bbec3d4ffa808a69bea6698dec` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.124944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4035b9bbec3d4ffa808a69bea6698dec)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.126846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.133918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:17.173552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4035b9bbec3d4ffa808a69bea6698dec', 'request_id': 'req-33286876-3ad6-4f72-81e4-df88dc5bc0c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:17.180102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4035b9bbec3d4ffa808a69bea6698dec', 'request_id': 'req-33286876-3ad6-4f72-81e4-df88dc5bc0c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:17.187256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4035b9bbec3d4ffa808a69bea6698dec', 'request_id': 'req-33286876-3ad6-4f72-81e4-df88dc5bc0c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:17.198524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] The token cache is being invalidate because project 4035b9bbec3d4ffa808a69bea6698dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:17.198809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:17.199431 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-33286876-3ad6-4f72-81e4-df88dc5bc0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:17.200969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1061/2129] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:17 2022] DELETE /identity/v3/projects/4035b9bbec3d4ffa808a69bea6698dec => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:17.205903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.206412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.206508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.225786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:17.226994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.267401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:17.290923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1069/2130] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:17.298666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.298950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.299226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.320226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:17.321427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.329688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1062/2131] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:17.333815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.344387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bbdef-a99f-4a2c-ac44-f7839b214674 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.350284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.350925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.351207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.351484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.352224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:17.352537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.354147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.366871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1070/2132] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:17.371414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.380794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95fe64-0de7-47d1-b8b6-0fac2107173a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.385615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.386241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.386541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.386799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.387474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.389409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.416685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1063/2133] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:17 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:17.421183 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.431963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae96b-f2e8-4106-beda-ffe4815a6567 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.492537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.493166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.493446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.493771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.494521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.496693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.518242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1071/2134] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:17 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:17.522427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.533171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20470576-fca1-492f-953a-cd1f3102ec6a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.538512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.539121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.539405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.539682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.540445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.542058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.552852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1064/2135] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:17.556876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.570972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de36-30cd-4b8b-8534-e6daa5f8b769 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.576241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.576912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.577196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.577581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.578250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.579970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5c192-d75e-4771-b8f5-865db2a733ca admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.591207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1072/2136] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:17.595353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.605899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa8ad-898e-41fc-a2ef-1407a00e64e4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.610514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.611167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.611450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.611727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] PATH_INFO: `/v3/projects/7ba4f58c70d24567b1e6d5e25d2c6ddb/users/693561110083462086517d748a11d81b/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.627182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba4f58c70d24567b1e6d5e25d2c6ddb, user_id=693561110083462086517d748a11d81b, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.629886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.641972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1065/2137] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:17 2022] PUT /identity/v3/projects/7ba4f58c70d24567b1e6d5e25d2c6ddb/users/693561110083462086517d748a11d81b/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:17.653492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.653781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.654063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.685207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] MFA Rules not processed for user `693561110083462086517d748a11d81b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:17.686568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.763137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1073/2138] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:17.774464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.776484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.843357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0b6d-5b57-495d-89d9-fa0c3110fb23 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:17.847538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:17.848179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:17.848456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:17.848722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:17.857047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:17.858691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035db4-12e1-4a6b-9378-8d3e0c86222b service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:17.868443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1066/2139] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:17.932290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:17.933696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:17.983931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:18.010771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697b50-2b43-42ba-ba3f-c2bc00c4b21e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:18.015520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:18.016210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.016473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.016750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] PATH_INFO: `/v3/users/3f17511222d54ede809be0b560df242f` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.017944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f17511222d54ede809be0b560df242f)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:18.020662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:18.069630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f17511222d54ede809be0b560df242f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.081693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f17511222d54ede809be0b560df242f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.103081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f17511222d54ede809be0b560df242f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.110365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f17511222d54ede809be0b560df242f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.111269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f17511222d54ede809be0b560df242f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.118701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1074/2140] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:17 2022] DELETE /identity/v3/users/3f17511222d54ede809be0b560df242f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:18.123434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.123908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.123986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.172200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:18.173568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:18.230500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:18.266826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1067/2141] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:18.277768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:18.279165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:18.337653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35780d-4a12-4c3c-b478-a1858f82f001 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:18.341394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:18.342021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.342302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.342585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.351922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:18.353365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20529f8-9ddb-4e5a-9038-5cbd603503e2 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:18.364988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1075/2142] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:18.641555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:18.643030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:18.661210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdcc24-5127-4768-b859-7492c11f814f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:18.667070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:18.667734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.668191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.668393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] PATH_INFO: `/v3/projects/b69b9aea60544dfe94ef7e2e3659a0b8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.669546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69b9aea60544dfe94ef7e2e3659a0b8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:18.671752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:18.733053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69b9aea60544dfe94ef7e2e3659a0b8', 'request_id': 'req-c9498cb0-c133-4b33-83d1-84d7664075c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.741348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69b9aea60544dfe94ef7e2e3659a0b8', 'request_id': 'req-c9498cb0-c133-4b33-83d1-84d7664075c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.749576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69b9aea60544dfe94ef7e2e3659a0b8', 'request_id': 'req-c9498cb0-c133-4b33-83d1-84d7664075c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.761418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] The token cache is being invalidate because project b69b9aea60544dfe94ef7e2e3659a0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:18.761702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.762330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c9498cb0-c133-4b33-83d1-84d7664075c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:18.763808 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1068/2143] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:18 2022] DELETE /identity/v3/projects/b69b9aea60544dfe94ef7e2e3659a0b8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:18.768571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.768856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.769131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.788450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:18.789667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:18.843203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:18.856555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1076/2144] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:18.866039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.866332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.866614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.887551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:18.889209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:18.897469 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1069/2145] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:18.901587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:18.912006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0314-6b51-4b9b-ac98-3b4f72ac5c55 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:18.916944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:18.917563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.917849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.918129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.918836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:18.919145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:18.920807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:18.934375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1077/2146] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:18.939634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:18.950170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f562662-b201-4cbc-b25a-3bdea856934e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:18.960398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:18.961437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:18.961973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:18.962454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:18.963499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:18.965582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:18.998829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1070/2147] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:18 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:19.002749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.013689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745f755-7292-483a-a8e9-e790755fc5ea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.079229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.079963 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.080244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.080518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.081990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.084162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.107569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1078/2148] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:19 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:19.111891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.122156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0dbd3-271a-4469-9b28-113cdfe28de8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.126795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.127400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.127665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.128494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.129186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.130762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.140586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1071/2149] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:19.145029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.155063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864323a-7c0d-4e64-8921-3c32c8d9f350 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.159763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.160401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.160686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.160971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.161668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.163265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2649805-bf9d-4873-b499-3e1998b56030 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.173979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1079/2150] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:19.178597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.188668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07032514-9b0a-41ab-8eae-7bf845c356d3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.194194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.194850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.195132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.195404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] PATH_INFO: `/v3/projects/be55da7bb17742f6ab816269f904260e/users/ee2e7ff2e2dd42dca4b88fa1800d6871/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.210254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be55da7bb17742f6ab816269f904260e, user_id=ee2e7ff2e2dd42dca4b88fa1800d6871, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.213100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.227662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1072/2151] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:19 2022] PUT /identity/v3/projects/be55da7bb17742f6ab816269f904260e/users/ee2e7ff2e2dd42dca4b88fa1800d6871/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:19.247787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.248189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.248539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.286526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] MFA Rules not processed for user `ee2e7ff2e2dd42dca4b88fa1800d6871`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:19.287818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:19.357904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1080/2152] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:19.362261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.421861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:19.446537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7ddad-85af-409e-a5d2-58ef8b27cd64 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.453655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.454335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.454802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.454925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.455581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.457598 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.490694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1073/2153] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:19 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:19.497152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.509068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028bd2-ef00-4e0c-b1c3-c46b8c833d0a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.564004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:19.577194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.578177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.578504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.579110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.580013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.582021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.606704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1081/2154] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:19 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:19.610737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.621447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada9527-031d-42bd-adb0-5dabfa4f513c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.626029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.626632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.626909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.627182 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.627897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.629507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.639901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1074/2155] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:19.644086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.654101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79439978-e5f4-4407-9301-1071549515ed admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.658577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.659723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.660031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.660304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] PATH_INFO: `/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.676185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3d00aa6b2f46429debb9076bbf7f2a, user_id=3f9cdc2f8f0d4eac846bbd6a6e3f73f6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.678827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.692373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1082/2156] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:19 2022] PUT /identity/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:19.696497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.707136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bf0ef-c61e-41f9-9044-58b8fdd1d30d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.766333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.766951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.767233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.767523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.768228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.769856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c158-cc2f-4b53-baaa-0b4b52efecd7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.781758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1075/2157] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:19.783902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.796870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdf9-c95f-4de8-9c49-a7cfe77df0e8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:19.805178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:19.806150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.806779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.806779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] PATH_INFO: `/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.809589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3d00aa6b2f46429debb9076bbf7f2a, user_id=3f9cdc2f8f0d4eac846bbd6a6e3f73f6, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:19.813933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:19.826521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1083/2158] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:19 2022] PUT /identity/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:19.838168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:19.838464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:19.838748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:19.867404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] MFA Rules not processed for user `3f9cdc2f8f0d4eac846bbd6a6e3f73f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:19.868656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:19.935050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1076/2159] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:19.946274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:19.948027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:20.004672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd7d68-7d00-40dd-9dba-85853a655511 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:20.008348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:20.009049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:20.009332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:20.009609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:20.020280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:20.021754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:20.032175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1084/2160] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:20.110453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:20.120095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62685cb8-e222-40ba-b42a-0376e13e53f0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:20.123723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:20.124420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:20.124706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:20.125107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:20.176595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:20.190164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:20.191591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d08c1d-6dae-45eb-a800-5aa3508ce2af service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:20.201361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1077/2161] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:28.636648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:28.638130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:28.712832 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24113b4-d821-4a83-8c49-167ccfba7e9a service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:28.718205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:28.718855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:28.719206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:28.719491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] PATH_INFO: `/v3/users/693561110083462086517d748a11d81b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:28.720702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=693561110083462086517d748a11d81b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:28.723167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:28.780882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '693561110083462086517d748a11d81b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:28.787085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:28.791596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '693561110083462086517d748a11d81b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:28.810558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '693561110083462086517d748a11d81b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:28.819146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '693561110083462086517d748a11d81b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:28.820809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '693561110083462086517d748a11d81b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:28.831167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1085/2162] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:28 2022] DELETE /identity/v3/users/693561110083462086517d748a11d81b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:28.836069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b4bb9-66d8-4cd7-9f48-e9da5f2147e0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:28.836961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b4bb9-66d8-4cd7-9f48-e9da5f2147e0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:28.837457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b4bb9-66d8-4cd7-9f48-e9da5f2147e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:28.862639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0a7b4bb9-66d8-4cd7-9f48-e9da5f2147e0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:28.864322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0a7b4bb9-66d8-4cd7-9f48-e9da5f2147e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:28.935894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1078/2163] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:28.950158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:28.951972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:29.043598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e228dcf-9f0d-429a-9777-50f3ab748480 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:29.055252 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:29.055993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.056422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.060268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.079660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a148-c3ec-4d9a-b3bc-4651b50e23e9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.079660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a148-c3ec-4d9a-b3bc-4651b50e23e9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.079660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a148-c3ec-4d9a-b3bc-4651b50e23e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.092211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:29.093690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:29.116250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bbd7a148-c3ec-4d9a-b3bc-4651b50e23e9 None None] MFA Rules not processed for user `d737eac8ae0343df9ace919d75596800`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:29.116426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1086/2164] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:29.118216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bbd7a148-c3ec-4d9a-b3bc-4651b50e23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:29.212511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1079/2165] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:29.622597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:29.624267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:29.644863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42695e3-1951-412f-b422-7346000b8640 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:29.665152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:29.666327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.666902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.667345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] PATH_INFO: `/v3/projects/7ba4f58c70d24567b1e6d5e25d2c6ddb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.669805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba4f58c70d24567b1e6d5e25d2c6ddb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:29.671818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:29.736363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba4f58c70d24567b1e6d5e25d2c6ddb', 'request_id': 'req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:29.743121 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba4f58c70d24567b1e6d5e25d2c6ddb', 'request_id': 'req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:29.757610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba4f58c70d24567b1e6d5e25d2c6ddb', 'request_id': 'req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:29.770515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] The token cache is being invalidate because project 7ba4f58c70d24567b1e6d5e25d2c6ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:29.770835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:29.771460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-28031b86-4302-4bf7-86f7-cb0d0d5d4b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:29.773087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1087/2166] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:29 2022] DELETE /identity/v3/projects/7ba4f58c70d24567b1e6d5e25d2c6ddb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:29.778341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.778631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.778907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.808515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:29.809814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:29.884567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1080/2167] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:29.893268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.893786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.894007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.916582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:29.917862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:29.926743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1088/2168] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:30:29.931109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:29.944302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515595c-6002-4cab-95e4-ffd4f9846be2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:29.949839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:29.950921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.951202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.951470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.952213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:29.952516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:29.954398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:29.967553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1081/2169] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:29.971762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:29.985716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2327a-f017-4bfe-b156-8c19792ef587 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:29.992796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:29.993407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:29.993688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:29.993959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:29.994642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:29.997428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.028553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1089/2170] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:29 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:30.032790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:30.045279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dfae1-229e-496e-a818-3c85e701bbed admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:30.071212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:30.119629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:30.120401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.120681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.120950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.121620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:30.123526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.147804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1082/2171] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:30 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:30:30.151845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:30.169736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9464a58-6b04-4c35-ba34-53aa7d8874b8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:30.176002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:30.177317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.177670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.178011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.178778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:30.180636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.195087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1090/2172] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:30.198789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:30.209774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747b9dd-2b72-4a52-bce9-fd87cb8a92d8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:30.214864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:30.215490 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.217626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.217924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.218638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:30.220473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b493cb-ee0d-46b3-8191-8b57c628fb28 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.236209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1083/2173] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:30.240202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:30.250847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e2e0-5bf1-4eb8-8800-17dfb7ed3bce admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:30.256285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:30.257198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.257483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.257760 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] PATH_INFO: `/v3/projects/8719c555ca834c738566a2cc7684e9eb/users/cd41786cce9c4b8081f3a466140d268b/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.275227 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:30.280603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8719c555ca834c738566a2cc7684e9eb, user_id=cd41786cce9c4b8081f3a466140d268b, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:30.283876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.302537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1091/2174] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:30 2022] PUT /identity/v3/projects/8719c555ca834c738566a2cc7684e9eb/users/cd41786cce9c4b8081f3a466140d268b/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:30:30.307524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274835-e6d5-4d02-84f4-52ea780a2663 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.308181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274835-e6d5-4d02-84f4-52ea780a2663 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.308610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274835-e6d5-4d02-84f4-52ea780a2663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.335818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cb274835-e6d5-4d02-84f4-52ea780a2663 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:30.337121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cb274835-e6d5-4d02-84f4-52ea780a2663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:30.481791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1084/2175] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:30.495407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:30.497595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:30.603661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9db7-b261-4171-8457-030df19118b9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:30.609123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:30.609881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:30.610168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:30.610449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:30.630266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:30.630629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:30.646012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1092/2176] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:31.603795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9269a3-83d0-4970-9b4f-495c5a636b67 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:31.604326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9269a3-83d0-4970-9b4f-495c5a636b67 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:31.604851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9269a3-83d0-4970-9b4f-495c5a636b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:31.652127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7e9269a3-83d0-4970-9b4f-495c5a636b67 None None] MFA Rules not processed for user `cd41786cce9c4b8081f3a466140d268b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:31.653358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e9269a3-83d0-4970-9b4f-495c5a636b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:31.780836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1085/2177] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:31.815092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:31.817367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:31.905620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c504791-b8b6-4b6e-8e7c-b2c8b09d0347 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:31.909776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:31.910581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:31.911013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:31.911499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:31.921804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:31.923365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:31.937046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1093/2178] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:33.629148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08194-15c8-4569-9e43-69ee3642d012 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:33.630163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08194-15c8-4569-9e43-69ee3642d012 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:33.630661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08194-15c8-4569-9e43-69ee3642d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:33.671537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4aa08194-15c8-4569-9e43-69ee3642d012 None None] MFA Rules not processed for user `cd41786cce9c4b8081f3a466140d268b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:33.671537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4aa08194-15c8-4569-9e43-69ee3642d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:33.686793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1086/2179] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:41.931646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:43.650610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:55.624694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:55.626128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:55.641680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc10d8-78cd-46fc-919d-7b2214814d31 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:55.646929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:55.647745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:55.648283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:55.648572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] PATH_INFO: `/v3/users/9648c91338aa4f4db619e0e371464e7a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:55.649759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9648c91338aa4f4db619e0e371464e7a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:55.651786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:55.707341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9648c91338aa4f4db619e0e371464e7a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:55.712355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9648c91338aa4f4db619e0e371464e7a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:55.725667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9648c91338aa4f4db619e0e371464e7a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:55.736062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9648c91338aa4f4db619e0e371464e7a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:55.736297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9648c91338aa4f4db619e0e371464e7a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:55.742964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1094/2180] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:30:55 2022] DELETE /identity/v3/users/9648c91338aa4f4db619e0e371464e7a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:55.747321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:55.747812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:55.748063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:55.770251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:55.771670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:55.865539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1087/2181] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:55.890031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:55.892248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:55.956204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ac0f-4f42-4150-80bb-51b850656423 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:55.959801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:55.960509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:55.960790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:55.961063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:55.971046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:55.972696 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffc8df-d7ee-4d36-a18f-a0d860078958 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:55.991324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1095/2182] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:30:56.267216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.268867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:56.283678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6fa96-107a-4cf2-a68a-9935ccafd620 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.289937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.290811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.291189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.291567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] PATH_INFO: `/v3/projects/382fa28f646949f0a53a537dd3a072a0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.293201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=382fa28f646949f0a53a537dd3a072a0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.295337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.348082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382fa28f646949f0a53a537dd3a072a0', 'request_id': 'req-119cd9f6-278e-43e2-87c7-b29f3fae49c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:56.354410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382fa28f646949f0a53a537dd3a072a0', 'request_id': 'req-119cd9f6-278e-43e2-87c7-b29f3fae49c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:56.362027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382fa28f646949f0a53a537dd3a072a0', 'request_id': 'req-119cd9f6-278e-43e2-87c7-b29f3fae49c4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:56.373130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] The token cache is being invalidate because project 382fa28f646949f0a53a537dd3a072a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:30:56.373417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:56.374055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-119cd9f6-278e-43e2-87c7-b29f3fae49c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:30:56.375582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1088/2183] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:30:56 2022] DELETE /identity/v3/projects/382fa28f646949f0a53a537dd3a072a0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:56.380461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.380748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.381027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.399988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:56.401281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:56.473698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1096/2184] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:56.481619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.481913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.482193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.505225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:56.506447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:56.515857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1089/2185] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:56.519343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.529910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985481-d86e-4299-a86f-fd5f5884db3d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.537515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.538345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.538561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.538765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.539465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:30:56.539770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.541394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.559584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1097/2186] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:30:56.560780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.571798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc354049-5964-4451-8e25-cda1cf7cf9af None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.577176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.577795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.578077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.578367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.579009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.580966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.613146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1090/2187] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:30:56 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:56.616802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.621973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:56.628324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa4dfa-3b25-471d-b23f-42a66e62640d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.688116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.688737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.689019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.689296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.689964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.691966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.716254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1098/2188] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:30:56 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:30:56.720341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.731170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ab75-8d60-47bf-9b42-e85958a62b1d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.735691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.736325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.736603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.736924 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.737713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.739277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.752129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1091/2189] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:30:56.754280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.765001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021d05-98d5-4f03-a97e-1c2661081a05 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.770454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.771060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.771338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.771611 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.772332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.773924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded4a6e-c3f9-429e-b373-19d61be71bc9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.785673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1099/2190] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:30:56.789942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.800696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078a34-6055-44cb-a932-692dd9aede04 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.807050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.807975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.808353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.808951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] PATH_INFO: `/v3/projects/8d6461094aaf4ef087513c7dd325487f/users/28becbddb21f4a32b7f8d4657bc0085e/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.830392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d6461094aaf4ef087513c7dd325487f, user_id=28becbddb21f4a32b7f8d4657bc0085e, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:56.833902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:56.850549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1092/2191] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:30:56 2022] PUT /identity/v3/projects/8d6461094aaf4ef087513c7dd325487f/users/28becbddb21f4a32b7f8d4657bc0085e/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:30:56.862343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167a539-592b-4b83-aa12-1a240058113f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.862644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167a539-592b-4b83-aa12-1a240058113f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.862932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167a539-592b-4b83-aa12-1a240058113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.888879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:56.890971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:56.892736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9167a539-592b-4b83-aa12-1a240058113f None None] MFA Rules not processed for user `28becbddb21f4a32b7f8d4657bc0085e`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:30:56.894627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9167a539-592b-4b83-aa12-1a240058113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:56.917016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:30:56.985664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89484e13-f52e-4536-aec7-582044326f65 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:56.989999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:56.991379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:56.991813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:56.992271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:56.995441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:57.000096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1100/2192] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:30:57.009702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167a539-592b-4b83-aa12-1a240058113f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:30:57.011459 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9167a539-592b-4b83-aa12-1a240058113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:30:57.093548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167a539-592b-4b83-aa12-1a240058113f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:30:57.097866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:30:57.098062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:57.098737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:30:57.099087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:30:57.099465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:30:57.099857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb6025-5459-4882-a85d-2f9b9006bf58 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:57.109040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:30:57.110609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:30:57.111861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1093/2193] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:30:57.121213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1101/2194] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:31:07.106530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=41 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:07.116352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:07.664997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfef8fb-c411-4dc2-81a1-bb03774b8e76 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:07.665532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfef8fb-c411-4dc2-81a1-bb03774b8e76 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:07.666023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfef8fb-c411-4dc2-81a1-bb03774b8e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:07.692886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fcfef8fb-c411-4dc2-81a1-bb03774b8e76 None None] MFA Rules not processed for user `51657a0e4865469999bdf8cd5102dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:07.694176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fcfef8fb-c411-4dc2-81a1-bb03774b8e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:07.748207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1094/2195] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:09.222379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:09.223928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:09.313895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a661-1ec8-4195-9a4c-ac64a5552768 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:09.324978 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:09.326930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:09.326930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:09.326930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] PATH_INFO: `/v3/users/ee2e7ff2e2dd42dca4b88fa1800d6871` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:09.327546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2e7ff2e2dd42dca4b88fa1800d6871)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:09.329540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:09.398097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2e7ff2e2dd42dca4b88fa1800d6871'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:09.413605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2e7ff2e2dd42dca4b88fa1800d6871'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:09.438351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2e7ff2e2dd42dca4b88fa1800d6871'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:09.464680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2e7ff2e2dd42dca4b88fa1800d6871'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:09.466101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2e7ff2e2dd42dca4b88fa1800d6871'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:09.472860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1102/2196] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:09 2022] DELETE /identity/v3/users/ee2e7ff2e2dd42dca4b88fa1800d6871 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:09.477516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:09.477968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:09.478069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:09.504084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:09.505323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:09.577819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1095/2197] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:31:09.592952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:09.594716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:09.688944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a67f-e249-4558-a171-ca6c2ec43e09 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:09.693468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:09.694088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:09.694372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:09.694644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:09.704793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:09.706203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:09.716597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1103/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:10.020345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.021899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:10.037266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fce337-a9ea-4a4c-95a7-41cdc2ccc247 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:10.041923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:10.042565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.042848 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.043127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] PATH_INFO: `/v3/projects/be55da7bb17742f6ab816269f904260e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.044534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be55da7bb17742f6ab816269f904260e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:10.046356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:10.092864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be55da7bb17742f6ab816269f904260e', 'request_id': 'req-9b808b43-e282-48f8-bd44-9477e7ad8596'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.100006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be55da7bb17742f6ab816269f904260e', 'request_id': 'req-9b808b43-e282-48f8-bd44-9477e7ad8596'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.108064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be55da7bb17742f6ab816269f904260e', 'request_id': 'req-9b808b43-e282-48f8-bd44-9477e7ad8596'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.120923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] The token cache is being invalidate because project be55da7bb17742f6ab816269f904260e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:10.121214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.121854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9b808b43-e282-48f8-bd44-9477e7ad8596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.123409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1096/2199] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:10 2022] DELETE /identity/v3/projects/be55da7bb17742f6ab816269f904260e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:10.127514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.137245 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103baad-e33b-43c0-9573-b211ff2aac66 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:10.155114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:10.194991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:10.195619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.195929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.196202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] PATH_INFO: `/v3/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.197348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9cdc2f8f0d4eac846bbd6a6e3f73f6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:10.199255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:10.244262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9cdc2f8f0d4eac846bbd6a6e3f73f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.251136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9cdc2f8f0d4eac846bbd6a6e3f73f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.262969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9cdc2f8f0d4eac846bbd6a6e3f73f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.270301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9cdc2f8f0d4eac846bbd6a6e3f73f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.270938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9cdc2f8f0d4eac846bbd6a6e3f73f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.276865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1104/2200] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:10 2022] DELETE /identity/v3/users/3f9cdc2f8f0d4eac846bbd6a6e3f73f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:10.508570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c645-52c8-4c90-b463-dd40f50be171 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.509100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c645-52c8-4c90-b463-dd40f50be171 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.509191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c645-52c8-4c90-b463-dd40f50be171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.529728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ff63c645-52c8-4c90-b463-dd40f50be171 None None] MFA Rules not processed for user `cd41786cce9c4b8081f3a466140d268b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:10.531297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ff63c645-52c8-4c90-b463-dd40f50be171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:10.544143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.545666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:10.563976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:10.613414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1097/2201] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:10.629099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b807b7-55fe-4f92-bab7-ce258c704e14 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:10.635048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:10.635704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.636016 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.636288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] PATH_INFO: `/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.637532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3d00aa6b2f46429debb9076bbf7f2a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:10.639680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:10.687706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:10.692515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3d00aa6b2f46429debb9076bbf7f2a', 'request_id': 'req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.700158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3d00aa6b2f46429debb9076bbf7f2a', 'request_id': 'req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.708277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3d00aa6b2f46429debb9076bbf7f2a', 'request_id': 'req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.720166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] The token cache is being invalidate because project 0e3d00aa6b2f46429debb9076bbf7f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:10.720451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.721661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0cda1fd6-9102-4ff2-acae-3adbe27f2a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:10.723593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1105/2202] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:10 2022] DELETE /identity/v3/projects/0e3d00aa6b2f46429debb9076bbf7f2a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:10.728365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.728654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.729100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.749872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:10.750998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:10.825419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1098/2203] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:10.833295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.833578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.833852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.854559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:10.855779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:10.864460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1106/2204] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:10.868660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.878471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b76bd5-f3ec-4fa7-94ff-45befcca9fae None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:10.883026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:10.883639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.884507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.884507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.885046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:31:10.885350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:10.887218 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:10.901451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1099/2205] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:31:10.905785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.915631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3469b6-1f1a-4077-aae6-d71bcd6d2430 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:10.920958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:10.921576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:10.921879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:10.922150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:10.922820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:10.924797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:10.956472 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1107/2206] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:10 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:10.960710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:10.971677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78397-5ef8-4ac9-9d34-36acc6dd95a0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.040629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.041293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.041572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.041841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.042493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.044451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.068398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1100/2207] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:10 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:11.071335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.082049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473157ff-ed84-448b-948d-d5ac6a2fa4b7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.086664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.087278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.087556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.087850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.088555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.090157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.100900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1108/2208] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:11.104895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.115873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6855-2e1f-4ebd-a0a4-9cd734e995c6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.120811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.121428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.121711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.121989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.122692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.124336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9b8f-6bf8-4fb8-9d55-e64d03a176ea admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.135752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1101/2209] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:11.140174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.150086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb06f-a28a-40f2-a255-8f1d53625797 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.154732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.155382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.155663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.155973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] PATH_INFO: `/v3/projects/197847dc6a964964b5b48a542410f04c/users/21ef6e76855d4999965bb42e5d2858eb/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.171193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=197847dc6a964964b5b48a542410f04c, user_id=21ef6e76855d4999965bb42e5d2858eb, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.173917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.188268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1109/2210] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:11 2022] PUT /identity/v3/projects/197847dc6a964964b5b48a542410f04c/users/21ef6e76855d4999965bb42e5d2858eb/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:11.201340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69d53-7684-4131-a994-61929166442f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.201857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69d53-7684-4131-a994-61929166442f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.202136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69d53-7684-4131-a994-61929166442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.232106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cbc69d53-7684-4131-a994-61929166442f None None] MFA Rules not processed for user `21ef6e76855d4999965bb42e5d2858eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:11.233351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cbc69d53-7684-4131-a994-61929166442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:11.271956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:11.312923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1102/2211] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:11.317929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.381179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304455-bc7f-464e-99da-f4857765f27c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.385871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.386486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.386766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.387042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.387706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.390063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.418048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1110/2212] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:11 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:31:11.422776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69d53-7684-4131-a994-61929166442f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.435630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69d53-7684-4131-a994-61929166442f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.500671 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.501475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.501827 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.502267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.503285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.506320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.532430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1103/2213] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:11 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:31:11.536861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.547764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813671a-eb00-4ca7-a9d2-965ec08d1afe admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.553205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.553817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.554105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.554367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.555061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.556690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.567805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1111/2214] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:11.572081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.582624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c84d7-c998-48a4-8bd3-eebe9a5fc21f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.587356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.588064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.588352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.588639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] PATH_INFO: `/v3/projects/281e17f826af4c599c35a86913e63919/users/376e9570fe8e4b5c9e2e8db6e2b05d97/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.603785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281e17f826af4c599c35a86913e63919, user_id=376e9570fe8e4b5c9e2e8db6e2b05d97, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.606466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.620518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1104/2215] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:11 2022] PUT /identity/v3/projects/281e17f826af4c599c35a86913e63919/users/376e9570fe8e4b5c9e2e8db6e2b05d97/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:11.624617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.634750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9360f-2743-4649-954e-9bf58fc30143 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.674730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:11.692457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.693065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.693345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.693619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.694312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.695930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3d5e3-1358-4e06-aa07-72d9c9ec801f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.707049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1112/2216] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:11.711198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.721905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95cda-e6be-4922-839d-f843a1c6c892 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.726475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.727183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.727464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.727739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] PATH_INFO: `/v3/projects/281e17f826af4c599c35a86913e63919/users/376e9570fe8e4b5c9e2e8db6e2b05d97/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.729701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281e17f826af4c599c35a86913e63919, user_id=376e9570fe8e4b5c9e2e8db6e2b05d97, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.732356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.743191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1105/2217] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:11 2022] PUT /identity/v3/projects/281e17f826af4c599c35a86913e63919/users/376e9570fe8e4b5c9e2e8db6e2b05d97/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:11.753301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.753594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.753879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.783276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] MFA Rules not processed for user `376e9570fe8e4b5c9e2e8db6e2b05d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:11.784999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:11.857527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1113/2218] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:11.868256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:11.870024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:11.927377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e384a19-920a-4506-a48f-092f266bad16 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:11.931034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:11.931630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:11.931936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:11.932210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:11.941256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:11.942941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b015db-3ce6-4228-9b1f-827e31eaed5f service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:11.959224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1106/2219] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:11 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:13.478094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:13.548162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b451255-b633-4d9e-8948-8b7298334a41 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:13.552822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:13.553502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:13.553895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:13.554072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] PATH_INFO: `/v3/users/21ef6e76855d4999965bb42e5d2858eb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:13.555242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ef6e76855d4999965bb42e5d2858eb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:13.557276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:13.632671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ef6e76855d4999965bb42e5d2858eb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:13.662787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ef6e76855d4999965bb42e5d2858eb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:13.672358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:13.702892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ef6e76855d4999965bb42e5d2858eb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:13.712356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ef6e76855d4999965bb42e5d2858eb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:13.713584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ef6e76855d4999965bb42e5d2858eb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:13.722888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1114/2220] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:13 2022] DELETE /identity/v3/users/21ef6e76855d4999965bb42e5d2858eb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:13.730959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:13.730959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:13.730959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:13.761720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:13.763725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:13.790340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:13.864313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1107/2221] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:13.888836 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:13.890983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:13.980764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d902d-1d61-42c0-89ce-5c164bed3c5a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:13.990001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:13.990520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:13.990802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:13.991083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:14.001503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:14.003578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:14.014510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1115/2222] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:14.976507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:14.978015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:15.000240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470c795-d2f8-4bfb-9ad4-faf7f766bf21 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:15.005737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:15.006565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:15.006841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:15.007287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] PATH_INFO: `/v3/projects/197847dc6a964964b5b48a542410f04c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:15.008505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=197847dc6a964964b5b48a542410f04c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:15.010443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:15.069690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197847dc6a964964b5b48a542410f04c', 'request_id': 'req-b93a682b-ac97-4c51-b11e-3518cfa9bcee'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.080116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197847dc6a964964b5b48a542410f04c', 'request_id': 'req-b93a682b-ac97-4c51-b11e-3518cfa9bcee'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.090041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197847dc6a964964b5b48a542410f04c', 'request_id': 'req-b93a682b-ac97-4c51-b11e-3518cfa9bcee'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.104521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] The token cache is being invalidate because project 197847dc6a964964b5b48a542410f04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:15.104925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.105698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.107315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1108/2223] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:14 2022] DELETE /identity/v3/projects/197847dc6a964964b5b48a542410f04c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:15.111551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:15.121439 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92a55b-1d6c-4330-8c28-18e2c0710997 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:15.201758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:15.202571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:15.204295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:15.204605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] PATH_INFO: `/v3/users/376e9570fe8e4b5c9e2e8db6e2b05d97` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:15.212855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376e9570fe8e4b5c9e2e8db6e2b05d97)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:15.215049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:15.290085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376e9570fe8e4b5c9e2e8db6e2b05d97'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.302418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376e9570fe8e4b5c9e2e8db6e2b05d97'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.324447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376e9570fe8e4b5c9e2e8db6e2b05d97'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.333360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376e9570fe8e4b5c9e2e8db6e2b05d97'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.334549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2ad39d53-7116-444b-a399-20de5960cf57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376e9570fe8e4b5c9e2e8db6e2b05d97'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:15.343492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1116/2224] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:15 2022] DELETE /identity/v3/users/376e9570fe8e4b5c9e2e8db6e2b05d97 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:16.170207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.173073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:16.190762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a682b-ac97-4c51-b11e-3518cfa9bcee admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.276908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.277930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.278282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.278625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] PATH_INFO: `/v3/projects/281e17f826af4c599c35a86913e63919` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.279980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281e17f826af4c599c35a86913e63919)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.281997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.333804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281e17f826af4c599c35a86913e63919', 'request_id': 'req-4ff0fcbe-3d74-4800-b2fb-28daeb745347'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:16.335034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:16.341815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281e17f826af4c599c35a86913e63919', 'request_id': 'req-4ff0fcbe-3d74-4800-b2fb-28daeb745347'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:16.350014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281e17f826af4c599c35a86913e63919', 'request_id': 'req-4ff0fcbe-3d74-4800-b2fb-28daeb745347'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:16.362435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] The token cache is being invalidate because project 281e17f826af4c599c35a86913e63919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:16.362723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:16.363383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4ff0fcbe-3d74-4800-b2fb-28daeb745347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:16.365181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1109/2225] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:16 2022] DELETE /identity/v3/projects/281e17f826af4c599c35a86913e63919 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:16.370050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.370547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.370713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.381128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:16.393681 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:16.394999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:16.458050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1117/2226] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:16.466000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.466504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.466784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.488785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:16.490332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:16.499787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1110/2227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:16.504634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.515041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77994163-fb42-4bfb-92bb-804c05938b69 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.519778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.520549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.520955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.521329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.522123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:31:16.522612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.524793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.538597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1118/2228] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:31:16.542565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.552582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ce890-3ea7-4ac6-8112-9a390e69f7bf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.557842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.558476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.558762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.559041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.559729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.561690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.596554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1111/2229] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:16.597738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.607849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335da7-7dfc-4e3d-9b8a-716d58a33469 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.669816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.670541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.670916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.671259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.672032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.674675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.697041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1119/2230] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:16.701117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.711616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697a201-92e9-4702-b9a1-57b9922cea4d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.716212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.716822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.717108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.717386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.718183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.719785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.730905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1112/2231] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:16.734652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.745540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467b9a-41bb-4a9c-b0cc-340a13b1a108 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.750425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.751044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.751327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.751604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.752367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.753973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ee6a2-d1f7-4f37-8f42-b12aebf8e186 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.764499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1120/2232] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:16.768725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:16.778844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088a6a9-b4a8-411e-a3db-f1f76e30f010 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:16.785214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:16.785909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.787794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.787794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] PATH_INFO: `/v3/projects/1dd0cdb21c9147cab25c268ea8a9bc9a/users/1bcabae9599e4e9b9e6c02948a38dd64/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.803455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd0cdb21c9147cab25c268ea8a9bc9a, user_id=1bcabae9599e4e9b9e6c02948a38dd64, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:16.806238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:16.830391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1113/2233] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:16 2022] PUT /identity/v3/projects/1dd0cdb21c9147cab25c268ea8a9bc9a/users/1bcabae9599e4e9b9e6c02948a38dd64/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:16.842451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:16.842796 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:16.843079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:16.876204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] MFA Rules not processed for user `1bcabae9599e4e9b9e6c02948a38dd64`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:16.878446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:16.963424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:16.967851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1121/2234] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:16.972868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.036925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81dffd-f4ca-429e-b9c0-6b9843257780 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.041901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.044281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.044569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.044849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.045529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.047853 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.076317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1114/2235] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:16 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:31:17.081110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.092514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379869f1-98b4-4203-a99f-9cf23336a263 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.152753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.153563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.154022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.154422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.155158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.157171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.178717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1122/2236] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:17 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:31:17.182550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.193189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89a75-1371-4df8-b3fb-569a7a57e4c2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.198193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.198834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.199117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.199395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.200119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.201717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.215011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1115/2237] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:17.217789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.228931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d499-b8ca-4771-968d-092ad63c0f8a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.233645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.234313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.234599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.234880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] PATH_INFO: `/v3/projects/b58039dcdaf045089baffa6790bf7b2f/users/b88a34c1aedd4db58855a107c52fed0f/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.251530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58039dcdaf045089baffa6790bf7b2f, user_id=b88a34c1aedd4db58855a107c52fed0f, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.254864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.270959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1123/2238] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:17 2022] PUT /identity/v3/projects/b58039dcdaf045089baffa6790bf7b2f/users/b88a34c1aedd4db58855a107c52fed0f/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:17.277301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.291439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be2b2-ad3c-49b8-ae57-0635186c3b5a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.339116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:17.354060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.354689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.354974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.355255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.356017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.357613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327f17c-25e4-4f5a-8e15-b3fc6cea7b6e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.368201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1116/2239] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:17.372443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.383365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04b7c1-0d7d-4fef-bb35-6d2d26f3de91 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.388050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.388714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.389003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.389279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] PATH_INFO: `/v3/projects/b58039dcdaf045089baffa6790bf7b2f/users/b88a34c1aedd4db58855a107c52fed0f/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.391235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58039dcdaf045089baffa6790bf7b2f, user_id=b88a34c1aedd4db58855a107c52fed0f, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.394509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.405177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1124/2240] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:17 2022] PUT /identity/v3/projects/b58039dcdaf045089baffa6790bf7b2f/users/b88a34c1aedd4db58855a107c52fed0f/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:17.415514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.415901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.416193 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.445529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] MFA Rules not processed for user `b88a34c1aedd4db58855a107c52fed0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:17.446731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:17.523127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1117/2241] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:31:17.536740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.538570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:17.603864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9ecd1-484c-4c97-8432-958b8333b91c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.608859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.609435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.610838 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.611138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.623671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.625226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.636573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1125/2242] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:17.740511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:17.750403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f5468-c13f-4653-8936-35a5a7f792a9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:17.756265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:17.757017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:17.757965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:17.758452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:17.824802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:17.826249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb1a1e-10f5-4bae-b2f8-c2c1e49a3c7c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:17.836046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1118/2243] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:19.079283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:19.080871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:19.184093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:19.200915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49fc8f-3c81-44ac-b267-c9ce3dcbdef7 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:19.209175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:19.210064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:19.210446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:19.210817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] PATH_INFO: `/v3/users/28becbddb21f4a32b7f8d4657bc0085e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:19.212231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28becbddb21f4a32b7f8d4657bc0085e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:19.214287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:19.324735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28becbddb21f4a32b7f8d4657bc0085e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:19.336362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28becbddb21f4a32b7f8d4657bc0085e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:19.355424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28becbddb21f4a32b7f8d4657bc0085e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:19.367089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28becbddb21f4a32b7f8d4657bc0085e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:19.368655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28becbddb21f4a32b7f8d4657bc0085e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:19.379283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1126/2244] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:19 2022] DELETE /identity/v3/users/28becbddb21f4a32b7f8d4657bc0085e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:31:19.384177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e742-7429-46a8-9380-4b15506476ab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:19.384951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e742-7429-46a8-9380-4b15506476ab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:19.385494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e742-7429-46a8-9380-4b15506476ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:19.423429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fe49e742-7429-46a8-9380-4b15506476ab None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:19.424810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fe49e742-7429-46a8-9380-4b15506476ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:19.581903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1119/2245] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:19.592010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:19.593443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:19.703139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76785-376e-4a4c-8861-a1afed9fe61f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:19.708794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:19.709618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:19.710097 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:19.710660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:19.724462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:19.726917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:19.743884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1127/2246] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:19.758231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:19.758520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:19.759078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:19.784102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:19.785327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:19.790205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:19.881117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1120/2247] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:19.934063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:19.936035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:20.016109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36c094-6740-45b7-ad07-435b50dda2f6 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:20.021317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:20.021941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:20.022228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:20.022510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:20.030704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:20.032327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:20.040704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1128/2248] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:31:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:31:20.119620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:20.121128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:20.161494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5199bf-f189-4d85-9240-1246305e40fd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:20.173563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:20.175194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:20.175622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:20.176196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] PATH_INFO: `/v3/projects/8d6461094aaf4ef087513c7dd325487f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:20.177611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d6461094aaf4ef087513c7dd325487f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:20.180517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:20.271650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d6461094aaf4ef087513c7dd325487f', 'request_id': 'req-feede60f-f59f-4e14-b82f-5cc6a92ec837'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:20.281585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d6461094aaf4ef087513c7dd325487f', 'request_id': 'req-feede60f-f59f-4e14-b82f-5cc6a92ec837'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:20.295506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d6461094aaf4ef087513c7dd325487f', 'request_id': 'req-feede60f-f59f-4e14-b82f-5cc6a92ec837'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:20.309667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] The token cache is being invalidate because project 8d6461094aaf4ef087513c7dd325487f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:20.309975 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:20.310687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-feede60f-f59f-4e14-b82f-5cc6a92ec837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:20.312245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1121/2249] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:20 2022] DELETE /identity/v3/projects/8d6461094aaf4ef087513c7dd325487f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:23.734030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:23.735945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:23.772792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5653ed-fe48-4f83-8900-af3b715d3285 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:23.797365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:23.877766 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:23.878744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:23.879021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:23.879294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] PATH_INFO: `/v3/users/1bcabae9599e4e9b9e6c02948a38dd64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:23.880679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcabae9599e4e9b9e6c02948a38dd64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:23.883029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:23.956851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcabae9599e4e9b9e6c02948a38dd64'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:23.974082 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcabae9599e4e9b9e6c02948a38dd64'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:23.990732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcabae9599e4e9b9e6c02948a38dd64'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:24.010268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcabae9599e4e9b9e6c02948a38dd64'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:24.011311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcabae9599e4e9b9e6c02948a38dd64'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:24.019772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1129/2250] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:23 2022] DELETE /identity/v3/users/1bcabae9599e4e9b9e6c02948a38dd64 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:24.024438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:24.025339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:24.025834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:24.072048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:24.073616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:24.173126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1122/2251] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:24.184047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:24.187148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:24.324032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d313c-dbaa-4a9a-8db3-6cac0d145192 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:24.333956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:24.336036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:24.336317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:24.336593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:24.367589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:24.369563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:24.387677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1130/2252] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:24.884251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:24.885777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:24.902954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d37ef-ce9c-443f-b708-4bd2697e199d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:24.909157 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:24.910243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:24.910536 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:24.910818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] PATH_INFO: `/v3/projects/1dd0cdb21c9147cab25c268ea8a9bc9a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:24.912806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd0cdb21c9147cab25c268ea8a9bc9a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:24.917146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:24.985106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:25.015183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd0cdb21c9147cab25c268ea8a9bc9a', 'request_id': 'req-10674161-c9a2-4fb1-abbd-82a5eb25797e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.026226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd0cdb21c9147cab25c268ea8a9bc9a', 'request_id': 'req-10674161-c9a2-4fb1-abbd-82a5eb25797e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.045410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd0cdb21c9147cab25c268ea8a9bc9a', 'request_id': 'req-10674161-c9a2-4fb1-abbd-82a5eb25797e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.067955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] The token cache is being invalidate because project 1dd0cdb21c9147cab25c268ea8a9bc9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:25.069131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.070789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.074281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1123/2253] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:24 2022] DELETE /identity/v3/projects/1dd0cdb21c9147cab25c268ea8a9bc9a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:25.080871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:25.102925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d8f4-7ba7-44f9-91cf-fb494f34ad02 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:25.140387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:25.191708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:25.193262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:25.194255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:25.194707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] PATH_INFO: `/v3/users/b88a34c1aedd4db58855a107c52fed0f` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:25.196877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b88a34c1aedd4db58855a107c52fed0f)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:25.202026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:25.286128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b88a34c1aedd4db58855a107c52fed0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.297129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b88a34c1aedd4db58855a107c52fed0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.331868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b88a34c1aedd4db58855a107c52fed0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.341602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b88a34c1aedd4db58855a107c52fed0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.342359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a4ca1b90-9ab2-4b2e-89e2-044c83567165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b88a34c1aedd4db58855a107c52fed0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:25.354897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1131/2254] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:25 2022] DELETE /identity/v3/users/b88a34c1aedd4db58855a107c52fed0f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:25.875427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:25.877333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:25.918900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674161-c9a2-4fb1-abbd-82a5eb25797e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.065983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.067314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.067782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.068417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] PATH_INFO: `/v3/projects/b58039dcdaf045089baffa6790bf7b2f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.070911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58039dcdaf045089baffa6790bf7b2f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.073538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.153410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58039dcdaf045089baffa6790bf7b2f', 'request_id': 'req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:26.169949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58039dcdaf045089baffa6790bf7b2f', 'request_id': 'req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:26.177196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58039dcdaf045089baffa6790bf7b2f', 'request_id': 'req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:26.199691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] The token cache is being invalidate because project b58039dcdaf045089baffa6790bf7b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:26.200285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:26.201429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f04ef1d1-3df1-46d7-a8d4-8605e260fe34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:26.206372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1124/2255] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:25 2022] DELETE /identity/v3/projects/b58039dcdaf045089baffa6790bf7b2f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:26.208204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.209159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.209856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.277386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:26.278687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:26.388685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1132/2256] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:26.402303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.402831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.403352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.436463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:26.438197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:26.459553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1125/2257] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:26.464434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.500123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2ea99-2b63-4447-acab-8dd98bd32fb9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.516904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.517799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.518154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.518498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.520735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:31:26.521218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.522937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.550403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1133/2258] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:31:26.561060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.585177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7467-45d8-44fd-8dd7-34d9cdeb7b26 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.593520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.594287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.594632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.594974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.596460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.599575 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.648456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1126/2259] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:26 2022] POST /identity/v3/projects => generated 357 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:26.654025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.669942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812c1d-f5a1-43b0-9327-48c36d999a31 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.748481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:26.801985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.802891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.803329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.803781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.804655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.806617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.843985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1134/2260] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:26 2022] POST /identity/v3/users => generated 279 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:26.850740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.865115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6ddb-213c-4d86-a07a-6fc65dcf7e5f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.873001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.873974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.874414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.875475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.876487 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.878550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.904426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1127/2261] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:26.910645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.924827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd93c5-c3b6-42c4-af77-d096b78949ba admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.933642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.935087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.936284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.936284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:26.937068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:26.938883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8367-3a6b-4c5e-8e07-7b9ff9ef8cc9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:26.957003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1135/2262] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:26.961412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:26.979736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5916a-95c8-4b79-ad0f-29edf9e4d025 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:26.985935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:26.986837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:26.987395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:26.987774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] PATH_INFO: `/v3/projects/c40acf201dfc4ce588eb390b319e89a9/users/5a573ce0766b4338b6feb4f10a873286/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:27.000192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:27.016197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40acf201dfc4ce588eb390b319e89a9, user_id=5a573ce0766b4338b6feb4f10a873286, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:27.021132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:27.069171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1128/2263] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:26 2022] PUT /identity/v3/projects/c40acf201dfc4ce588eb390b319e89a9/users/5a573ce0766b4338b6feb4f10a873286/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:27.080944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:27.081475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:27.081919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:27.208631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] MFA Rules not processed for user `5a573ce0766b4338b6feb4f10a873286`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:27.210111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:27.354736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1136/2264] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:27.409644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:27.418174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:27.530601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b36f-0c4b-4297-8e2b-7f50bc05e841 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:27.542159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:27.542812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:27.543094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:27.543367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:27.553552 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:27.555172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880627ff-4512-4cbd-9a2a-0c7a33e35439 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:27.566961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1129/2265] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:27.587264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:27.683671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a8fb3-3f93-454d-ba6e-bf169391c0e0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:27.690008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:27.690911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:27.691609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:27.692101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] PATH_INFO: `/v3/users/5a573ce0766b4338b6feb4f10a873286` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:27.693698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a573ce0766b4338b6feb4f10a873286)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:27.695667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:27.790807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:27.792010 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a573ce0766b4338b6feb4f10a873286'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:27.817014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a573ce0766b4338b6feb4f10a873286'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:27.855473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a573ce0766b4338b6feb4f10a873286'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:27.874224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a573ce0766b4338b6feb4f10a873286'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:27.876205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a573ce0766b4338b6feb4f10a873286'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:27.887795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1137/2266] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:31:27 2022] DELETE /identity/v3/users/5a573ce0766b4338b6feb4f10a873286 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:27.893276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:27.893568 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:27.893870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:27.935134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:27.936412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:28.065149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1130/2267] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:28.080763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:28.082346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:28.208069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0386ab-cbeb-41f1-8c30-3ac52721016a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:28.217476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:28.218933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:28.219384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:28.219854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:28.231268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:28.232722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325b0b1-142f-4f82-9264-61703f2dd961 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:28.245579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1138/2268] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:28.817361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:28.820793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:28.856420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf33d01-2e21-4d7d-a033-478e731d8459 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:28.873973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:28.876646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:28.878267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:28.878647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] PATH_INFO: `/v3/projects/c40acf201dfc4ce588eb390b319e89a9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:28.881167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40acf201dfc4ce588eb390b319e89a9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:28.886963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:28.980132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40acf201dfc4ce588eb390b319e89a9', 'request_id': 'req-55a2265b-8e52-4c67-b460-0ecc5fd7d367'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:28.991704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40acf201dfc4ce588eb390b319e89a9', 'request_id': 'req-55a2265b-8e52-4c67-b460-0ecc5fd7d367'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:29.004608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40acf201dfc4ce588eb390b319e89a9', 'request_id': 'req-55a2265b-8e52-4c67-b460-0ecc5fd7d367'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:29.028930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] The token cache is being invalidate because project c40acf201dfc4ce588eb390b319e89a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:31:29.029374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:29.033675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-55a2265b-8e52-4c67-b460-0ecc5fd7d367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:31:29.037418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1131/2269] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:31:28 2022] DELETE /identity/v3/projects/c40acf201dfc4ce588eb390b319e89a9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:29.042767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.043036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.043387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.070964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:29.072312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:29.181930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1139/2270] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:29.191787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.192223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.192606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.214364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:29.215755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:29.227041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1132/2271] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:29.232455 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.243526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadbe51-bd34-4eaa-bc78-fd380e569eb9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.249557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.250331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.250830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.251205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.252209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:31:29.252764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.254833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.274722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.281136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1140/2272] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:31:29.285925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6000f-11bd-47d5-a430-0d8cc53e35a4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.292449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.299660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.300000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.300315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.301512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.310002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.330420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:29.367115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1133/2273] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:31:29 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:31:29.371318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.383485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b33f-e96a-44e2-a90c-83d3fde66297 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.452535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:29.468951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.469936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.471157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.471687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.472709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.474948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.502757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1141/2274] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:31:29 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:31:29.506675 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.530678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52ce3f-2803-4137-b78d-cc0691d86644 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.537609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.538259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.538544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.538819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.539537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.541210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.552679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1134/2275] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:31:29.556733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.572207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156618-a642-4a3e-b22d-8f407600a330 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.587702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.587702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.588088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.588497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.589743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.592763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59be1-9668-4b65-b368-953446c4f0d2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.607386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1142/2276] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:31:29.611353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.622825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3992-92a2-42b2-af5c-d2b18f03efa1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.628100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.631406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.631696 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.632001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] PATH_INFO: `/v3/projects/ec0f892788aa4295b78d208c27363823/users/d21e589281b94e2c9a73a3030cb5425f/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.650290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0f892788aa4295b78d208c27363823, user_id=d21e589281b94e2c9a73a3030cb5425f, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.653183 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.670804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1135/2277] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:31:29 2022] PUT /identity/v3/projects/ec0f892788aa4295b78d208c27363823/users/d21e589281b94e2c9a73a3030cb5425f/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:31:29.681935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.685058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.685710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.727969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] MFA Rules not processed for user `d21e589281b94e2c9a73a3030cb5425f`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:29.729197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:29.854555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1143/2278] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:29.869202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:29.871672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:29.946015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b5f0-6b75-4a1f-bc29-61cef77cc261 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:29.956718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:29.957648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:29.958167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:29.958644 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:29.974514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:29.976911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc7d7e-6acf-4cf5-9f68-1b052d8e90e4 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:29.998612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1136/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:38.884468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:38.886290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:38.972694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9c4a1-02c6-4018-af4d-a4175501de18 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:38.978465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:38.979093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:38.979378 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:38.979652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:38.981449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:38.994399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:38.995945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:39.007004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1144/2280] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:39.755359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f9a8-af46-499e-97bc-7cfae50eb57b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:39.755873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f9a8-af46-499e-97bc-7cfae50eb57b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:39.756017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f9a8-af46-499e-97bc-7cfae50eb57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:39.776746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b408f9a8-af46-499e-97bc-7cfae50eb57b None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:39.777960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b408f9a8-af46-499e-97bc-7cfae50eb57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:39.837538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1137/2281] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:31:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:39.849876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:39.851307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:39.902153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:39.922043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ed21-3720-4519-aa84-078941a0b477 service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:39.929778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:39.929778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:39.929778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:39.929778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:39.938262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:39.940443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6b6b-132a-4075-906d-1a7fe5b7a52c service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:39.956846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1145/2282] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:31:40.876679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582804-eda2-4c69-baa0-53c41c57ddd7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:40.877240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582804-eda2-4c69-baa0-53c41c57ddd7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:40.877240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582804-eda2-4c69-baa0-53c41c57ddd7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:40.878746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1138/2283] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:31:40.890236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49040f-b460-4325-8660-573b4605230a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:40.890976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49040f-b460-4325-8660-573b4605230a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:40.892078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49040f-b460-4325-8660-573b4605230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:40.914155 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ee49040f-b460-4325-8660-573b4605230a None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:40.915740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ee49040f-b460-4325-8660-573b4605230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:41.041801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1146/2284] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:47.557723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e582804-eda2-4c69-baa0-53c41c57ddd7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:47.570088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e582804-eda2-4c69-baa0-53c41c57ddd7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:47.573938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:47.574559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.574845 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.575124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:47.587391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:47.588865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7841c7-a2a1-47a5-84f2-fa38bb161e91 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:47.609321 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1139/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:47.689538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.690234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.690381 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:47.692211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1147/2286] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:31:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 15:31:47.698834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9404a9-33ec-4870-9f08-a9c81fd5311b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.699135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9404a9-33ec-4870-9f08-a9c81fd5311b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.699419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9404a9-33ec-4870-9f08-a9c81fd5311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:47.720091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0c9404a9-33ec-4870-9f08-a9c81fd5311b None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:47.721319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0c9404a9-33ec-4870-9f08-a9c81fd5311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:47.771695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:31:47.786773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1140/2287] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:47.799331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:47.801179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:47.818997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedf9a7-f05b-4305-a4de-9c82c4260f6a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:47.824374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:47.825330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.825725 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.826007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:47.837175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:47.838916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aeaf5-0689-41c2-a672-c5e308af2727 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:47.847480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1148/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:47.960026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd9e66-53dd-4228-acfa-3b288d1aa043 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.960796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd9e66-53dd-4228-acfa-3b288d1aa043 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.961156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd9e66-53dd-4228-acfa-3b288d1aa043 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:47.962751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1141/2289] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:31:47.978863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6330-18ff-412d-a574-54caf2b5ef01 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:47.979151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6330-18ff-412d-a574-54caf2b5ef01 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:47.979435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6330-18ff-412d-a574-54caf2b5ef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:48.003977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-879f6330-18ff-412d-a574-54caf2b5ef01 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:48.005341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-879f6330-18ff-412d-a574-54caf2b5ef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:48.014865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1149/2290] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:31:48.025436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd9e66-53dd-4228-acfa-3b288d1aa043 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:48.036936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd9e66-53dd-4228-acfa-3b288d1aa043 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:48.040413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:48.041019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:48.041302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:48.041578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:48.052555 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:48.054048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:48.065514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1142/2291] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:31:51.162315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271302-c1af-49a8-a9d7-e2f01a8c7827 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:51.163201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271302-c1af-49a8-a9d7-e2f01a8c7827 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:51.163630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271302-c1af-49a8-a9d7-e2f01a8c7827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:51.184463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-45271302-c1af-49a8-a9d7-e2f01a8c7827 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:31:51.185764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-45271302-c1af-49a8-a9d7-e2f01a8c7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:31:51.194516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1150/2292] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:31:51.214723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:31:51.226712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6bdf-621e-47b2-8c66-da6b67f63ada service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:31:51.232814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:31:51.233455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:31:51.233736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:31:51.234012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:31:51.244797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:31:51.246721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:31:51.257613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1143/2293] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:32:01.179110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=52 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:01.253661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=15 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:02.527076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619803-0320-4660-b508-93c51ced7553 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:02.527542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619803-0320-4660-b508-93c51ced7553 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:02.527628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619803-0320-4660-b508-93c51ced7553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:02.548350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-db619803-0320-4660-b508-93c51ced7553 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:02.549563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-db619803-0320-4660-b508-93c51ced7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:02.556790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1151/2294] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:32:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:02.570036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:02.581281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d826-1554-4435-aa34-73c84c8c3ada service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:02.586367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:02.586984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:02.587267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:02.587543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:02.595774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:02.597375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:02.607013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1144/2295] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:32:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 15:32:04.607982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619803-0320-4660-b508-93c51ced7553 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:04.610934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-db619803-0320-4660-b508-93c51ced7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:04.682115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619803-0320-4660-b508-93c51ced7553 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:04.686850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:04.687689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:04.688019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:04.688298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] PATH_INFO: `/v3/users/cd41786cce9c4b8081f3a466140d268b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:04.689501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd41786cce9c4b8081f3a466140d268b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:04.691719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:04.749531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd41786cce9c4b8081f3a466140d268b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:04.758125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd41786cce9c4b8081f3a466140d268b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:04.770432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd41786cce9c4b8081f3a466140d268b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:04.777567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd41786cce9c4b8081f3a466140d268b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:04.778274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbb1b6f4-abcd-462f-bc82-17e7e3110f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd41786cce9c4b8081f3a466140d268b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:04.785067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1152/2296] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:32:04 2022] DELETE /identity/v3/users/cd41786cce9c4b8081f3a466140d268b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:04.958199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:04.970012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad948a8-2a4a-44ea-9857-66a3097d1851 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.030203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.031543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.031809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.032173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] PATH_INFO: `/v3/projects/8719c555ca834c738566a2cc7684e9eb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.033426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8719c555ca834c738566a2cc7684e9eb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.035420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.083002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8719c555ca834c738566a2cc7684e9eb', 'request_id': 'req-2890acef-7615-48b6-9b86-5efad5e3b426'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.089807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8719c555ca834c738566a2cc7684e9eb', 'request_id': 'req-2890acef-7615-48b6-9b86-5efad5e3b426'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.097943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8719c555ca834c738566a2cc7684e9eb', 'request_id': 'req-2890acef-7615-48b6-9b86-5efad5e3b426'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.108307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] The token cache is being invalidate because project 8719c555ca834c738566a2cc7684e9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:32:05.108592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.109214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2890acef-7615-48b6-9b86-5efad5e3b426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.110705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1145/2297] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:32:04 2022] DELETE /identity/v3/projects/8719c555ca834c738566a2cc7684e9eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:05.115959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.116247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.116528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.137423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:05.138617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:05.198684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1153/2298] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:05.206169 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.206460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.206736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.227398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:05.228639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:05.236744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1146/2299] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:05.241042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.251992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c390482-96b9-4c13-ae70-500d4c8cbe83 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.257782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.258410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.258693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.258968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.259665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:32:05.260011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.261748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.271793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:05.275449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1154/2300] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:32:05.279905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.291892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d48689-b2d9-48df-ad0b-4ce74bf2d97f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.296989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.297761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.298021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.298302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.299007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.300935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.335080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1147/2301] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:32:05.336269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.347200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966fff4-c56f-4e09-9a66-f71597474dd3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.404225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.404849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.405132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.405421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.406099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.408056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.428368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1155/2302] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:32:05.432681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.443466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf90b5e-021a-4829-9e1b-4af3190a5e33 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.448246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.448887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.449173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.449475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.450173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.451761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.462407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1148/2303] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:32:05.466205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.476467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb093f8-0b69-4384-85d2-f9ba11acade2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.481259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.481867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.482168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.482450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.483141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.484828 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e309a3d-aa97-4e3e-99e5-31506a005b29 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.495521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1156/2304] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:32:05.500435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.511202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46064-67ba-423a-b2f6-1c870b649ce8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.517933 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.518905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.519242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.519518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] PATH_INFO: `/v3/projects/a4a1d8782d0244d592cb1360c51e3bd8/users/ce6460a3cf0441abb7cc65774a18320d/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.535032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a1d8782d0244d592cb1360c51e3bd8, user_id=ce6460a3cf0441abb7cc65774a18320d, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.537833 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.551514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1149/2305] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:32:05 2022] PUT /identity/v3/projects/a4a1d8782d0244d592cb1360c51e3bd8/users/ce6460a3cf0441abb7cc65774a18320d/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:05.562877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.563160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.563450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.595152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] MFA Rules not processed for user `ce6460a3cf0441abb7cc65774a18320d`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:05.596340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:05.662070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1157/2306] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:32:05.673139 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.674867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:05.689418 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:05.734788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b18193-1782-463b-8d3d-106b36c2788c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.738390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.739014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.739358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.739631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.748311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.749725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7640f5d-c037-4fdf-871d-7c9899178245 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.758622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1150/2307] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:32:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:32:05.786414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:05.857886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83268621-038d-4494-beed-e21f4eb26ee3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:05.862546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:05.863200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.863481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.863757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] PATH_INFO: `/v3/users/ce6460a3cf0441abb7cc65774a18320d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.865328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6460a3cf0441abb7cc65774a18320d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:05.867493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:05.878729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:05.920822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6460a3cf0441abb7cc65774a18320d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.933228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6460a3cf0441abb7cc65774a18320d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.946407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6460a3cf0441abb7cc65774a18320d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.953834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6460a3cf0441abb7cc65774a18320d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.954534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6460a3cf0441abb7cc65774a18320d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:05.961140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1158/2308] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:32:05 2022] DELETE /identity/v3/users/ce6460a3cf0441abb7cc65774a18320d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:05.965556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:05.966034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:05.966133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:05.986376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:05.987604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.047289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1151/2309] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:06.057046 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:06.058467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.119191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fa1e5-a211-41f9-b213-4d9ea64beeeb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:06.122714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:06.123324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:06.123604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:06.123903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:06.133533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:06.134967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:06.145026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1159/2310] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:32:06.377267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:06.378674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.390478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbee128-23ea-4743-8730-d9b6df02ebc1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:06.395359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:06.396101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:06.396391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:06.396665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] PATH_INFO: `/v3/users/d21e589281b94e2c9a73a3030cb5425f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:06.397842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21e589281b94e2c9a73a3030cb5425f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:06.399793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:06.422603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:06.424733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.440129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cd630-4d55-4754-bf2d-20caa02e46e1 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:06.446370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:06.447013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:06.447379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:06.447857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] PATH_INFO: `/v3/projects/a4a1d8782d0244d592cb1360c51e3bd8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:06.449341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a1d8782d0244d592cb1360c51e3bd8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:06.451585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:06.457318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21e589281b94e2c9a73a3030cb5425f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.457989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21e589281b94e2c9a73a3030cb5425f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.466761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21e589281b94e2c9a73a3030cb5425f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.474623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21e589281b94e2c9a73a3030cb5425f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.488312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d52c68e5-af8c-45d5-9116-9d31612639f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21e589281b94e2c9a73a3030cb5425f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.495573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1152/2311] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:32:06 2022] DELETE /identity/v3/users/d21e589281b94e2c9a73a3030cb5425f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:06.500595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:06.500883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:06.501162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:06.506382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a1d8782d0244d592cb1360c51e3bd8', 'request_id': 'req-fe3b6147-935d-4494-9d28-b798530291d6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.512875 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a1d8782d0244d592cb1360c51e3bd8', 'request_id': 'req-fe3b6147-935d-4494-9d28-b798530291d6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.520297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:06.521159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a1d8782d0244d592cb1360c51e3bd8', 'request_id': 'req-fe3b6147-935d-4494-9d28-b798530291d6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.521703 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.533505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] The token cache is being invalidate because project a4a1d8782d0244d592cb1360c51e3bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:32:06.533795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.534431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:06.535972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1160/2312] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:32:06 2022] DELETE /identity/v3/projects/a4a1d8782d0244d592cb1360c51e3bd8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:06.548149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:06.589466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1153/2313] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:32:06.599568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:06.601197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:06.618093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:06.668715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6147-935d-4494-9d28-b798530291d6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:06.724370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:06.725035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:06.725314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:06.725589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:06.736375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:06.737844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb091e-5d94-45c5-a45b-8d04a408aa97 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:06.748505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1161/2314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:32:07.090104 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.091877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:07.104811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3b162-689d-4879-a76b-bc71a4d1c5db None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.109743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.110396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.110683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.111467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] PATH_INFO: `/v3/projects/ec0f892788aa4295b78d208c27363823` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.112999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0f892788aa4295b78d208c27363823)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.114947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.169797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0f892788aa4295b78d208c27363823', 'request_id': 'req-281f2fe7-889a-4a33-94b0-0866547e9c0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:07.176008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0f892788aa4295b78d208c27363823', 'request_id': 'req-281f2fe7-889a-4a33-94b0-0866547e9c0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:07.183342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0f892788aa4295b78d208c27363823', 'request_id': 'req-281f2fe7-889a-4a33-94b0-0866547e9c0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:07.194048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] The token cache is being invalidate because project ec0f892788aa4295b78d208c27363823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:32:07.194333 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:07.194965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-281f2fe7-889a-4a33-94b0-0866547e9c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:32:07.196519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1154/2315] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:32:07 2022] DELETE /identity/v3/projects/ec0f892788aa4295b78d208c27363823 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:07.201325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.201616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.201896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.224500 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:07.225699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:07.283508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1162/2316] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:07.291304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.291615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.291937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.311642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:07.312881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:07.321272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1155/2317] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:07.325997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.338013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94f64f-9d55-4acb-b8c2-5e408afb18fd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.343100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.344438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.344697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.344980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.345719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:32:07.346025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.347680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.359672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1163/2318] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:32:07.364058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.373283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ae9b-a659-4d2a-b6ae-6366794ccbd0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.377912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.378508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.378789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.379059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.379821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.381721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.410390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1156/2319] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:32:07 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:32:07.414520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.424494 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a32e-c8e8-4a16-bc54-bf2745dc6400 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.480955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.481498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.481782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.482062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.482736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.484719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.492604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:07.506157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1164/2320] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:32:07 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:32:07.510278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.521506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176739-f699-4107-aa83-c8c8c101d927 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.526257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.526861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.527137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.527406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.528136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.529710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.539935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1157/2321] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:32:07.543739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.554096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef296c35-50d0-4fd8-84a8-c1204f0bde7c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.558720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.559340 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.559622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.559922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.560615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.562777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0dc18-0cfa-4669-bf9d-50a98d719536 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.573756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1165/2322] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:32:07.578585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.588759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021efc12-5da9-4bbf-8675-8cce7946d5d8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.593957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.594893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.595185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.595459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] PATH_INFO: `/v3/projects/465f32cb4f724c18aad10fca114a1321/users/d040957e3ea04505bfa9e3b3d7c66f52/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.610688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=465f32cb4f724c18aad10fca114a1321, user_id=d040957e3ea04505bfa9e3b3d7c66f52, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.613341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.626524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1158/2323] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:32:07 2022] PUT /identity/v3/projects/465f32cb4f724c18aad10fca114a1321/users/d040957e3ea04505bfa9e3b3d7c66f52/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:32:07.632270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60bc4b-dd8d-4725-9956-27b97e05117e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.632667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60bc4b-dd8d-4725-9956-27b97e05117e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.633040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60bc4b-dd8d-4725-9956-27b97e05117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.652791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be60bc4b-dd8d-4725-9956-27b97e05117e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:07.654003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be60bc4b-dd8d-4725-9956-27b97e05117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:07.721583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1166/2324] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:07.731234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:07.733089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:07.788078 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:07.792142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9e227-9ddb-4729-8960-59ff0ec44cfc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:07.796834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:07.797499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:07.797928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:07.798447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:07.808075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:07.809512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:07.820145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1159/2325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:32:08.532847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537cd26-b780-4397-bfa1-0416278051c8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:08.533084 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537cd26-b780-4397-bfa1-0416278051c8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:08.533318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537cd26-b780-4397-bfa1-0416278051c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:08.567135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2537cd26-b780-4397-bfa1-0416278051c8 None None] MFA Rules not processed for user `d040957e3ea04505bfa9e3b3d7c66f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:08.568599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2537cd26-b780-4397-bfa1-0416278051c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:08.639390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1167/2326] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:08.650449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:08.651936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:08.712922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14a9b-49cf-48fe-90fb-da6a183441bc service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:08.716501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:08.717109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:08.717385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:08.717656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:08.727818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:08.729439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0201-9629-441a-8216-b7edd88a1131 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:08.739281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1160/2327] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:32:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:32:10.154724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:10.155264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:10.155264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:10.174711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] MFA Rules not processed for user `d040957e3ea04505bfa9e3b3d7c66f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:10.175951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:10.183115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1168/2328] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:32:15.999105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3dab4-3467-4c5f-8c4a-5ac3b7f51d97 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:15.999605 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3dab4-3467-4c5f-8c4a-5ac3b7f51d97 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:15.999724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3dab4-3467-4c5f-8c4a-5ac3b7f51d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:16.021992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7be3dab4-3467-4c5f-8c4a-5ac3b7f51d97 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:16.023626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7be3dab4-3467-4c5f-8c4a-5ac3b7f51d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:16.084819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1161/2329] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:32:16.097529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:32:16.099729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:16.165423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:16.172788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354e330-06c3-4e3d-a10f-05e53ac51683 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:32:16.179887 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:32:16.180526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:16.180813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:16.181093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:16.190812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:32:16.192474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de582d23-ccad-46a4-b456-5e5110cb964b service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:32:16.201197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1169/2330] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:32:26.077709 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=55 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:26.197691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:32:56.915515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de783f6-62e9-479c-8f91-434328537179 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:32:56.922223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de783f6-62e9-479c-8f91-434328537179 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:32:56.922223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de783f6-62e9-479c-8f91-434328537179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:32:56.936905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0de783f6-62e9-479c-8f91-434328537179 None None] MFA Rules not processed for user `51657a0e4865469999bdf8cd5102dad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:32:56.938405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0de783f6-62e9-479c-8f91-434328537179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:32:57.007111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1162/2331] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:33:06.999521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=25 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:33:18.270449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:18.270957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:18.271057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:18.290142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] MFA Rules not processed for user `d040957e3ea04505bfa9e3b3d7c66f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:33:18.291353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:18.298473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1170/2332] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:33:25.905395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:25.905866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:25.905994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:25.925673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] MFA Rules not processed for user `7a964a003f8e4f1da603a13eb7f534ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:33:25.926885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:25.994872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1163/2333] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:33:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:33:28.284287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:33:28.971537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:33:28.973013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:28.986418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c51b15-ccee-4f52-969c-cb590aa9dcb7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:33:28.990008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:33:28.990622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:28.990898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:28.991169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:28.992920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:29.069138 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:33:29.070510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c5a36-78e9-4fcc-9afd-81f4552f286e service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:33:29.082517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1171/2334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:33:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:33:30.277631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:33:30.288253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156c503-d274-4785-bb1c-eb22f033e077 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:33:30.292065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:33:30.292692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:30.292969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:30.293268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:30.295016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:30.311136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:33:30.312923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:33:30.329215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1164/2335] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:33:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:33:38.204562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:38.205152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:38.205152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:38.228295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] MFA Rules not processed for user `d737eac8ae0343df9ace919d75596800`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:33:38.229530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:38.288741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1172/2336] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:33:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:33:38.656168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:33:38.666106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee594a31-c966-4f90-9669-7e1b82a79c3b service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:33:38.669963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:33:38.670586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:38.670867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:38.671173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:38.673178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:38.688592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:33:38.690203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:33:38.699750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1165/2337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:33:48.280412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=59 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:33:48.695263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=37 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:33:59.457267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:33:59.469399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65c0e-700e-48b7-91ff-0957c8ebfdf1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:33:59.471984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:33:59.472556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:59.472833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:59.473104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:59.475055 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:59.488916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:33:59.491196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:33:59.511720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1173/2338] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:33:59.851994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:33:59.864698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae744d-6217-4866-9f89-d81651bd566d service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:33:59.869997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:33:59.870433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:33:59.870979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:33:59.871426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:33:59.874557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:33:59.890920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:33:59.892524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:33:59.902950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1166/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:34:04.991716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:04.993912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:05.012159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10e2d-4ff7-4227-9215-96e971888d54 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:05.022168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:05.023775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:05.024336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:05.024719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:05.026891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:05.040657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:05.042552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:05.054152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1174/2340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:05.824538 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:05.826463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:05.839521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5f8d3-b662-49a3-8391-6a9879cb1d21 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:05.845317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:05.846338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:05.846854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:05.847611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:05.851116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:05.924428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:05.925882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb23382-3d78-4d25-85ae-d0533e8b1e26 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:05.936518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1167/2341] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:10.026255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:10.047784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc577ace-8591-4a8a-9924-8ef86c170b22 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:10.074917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:10.075919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:10.076373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:10.076824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:10.079895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:10.100080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:10.103576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:10.119181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1175/2342] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:14.180293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f6c7-e655-4bcd-8d71-40552d5d43ce None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:14.180850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f6c7-e655-4bcd-8d71-40552d5d43ce None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:14.180850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f6c7-e655-4bcd-8d71-40552d5d43ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:14.201006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2464f6c7-e655-4bcd-8d71-40552d5d43ce None None] MFA Rules not processed for user `a2b027a2c8dd4459b1309faab28f3d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:14.202219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2464f6c7-e655-4bcd-8d71-40552d5d43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:14.209070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1168/2343] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:14.817754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:14.819953 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:14.908002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577331-0828-4791-9855-daa8fb2db76b service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:14.913057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:14.913866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:14.914164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:14.914427 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:14.917025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:14.929328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:14.930944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145f1c9-7c7c-436d-aeb4-4646163fbf15 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:14.941846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1176/2344] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:17.953001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62d9e6-43c0-4a61-908c-62903d78032e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:17.954040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62d9e6-43c0-4a61-908c-62903d78032e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:17.954319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62d9e6-43c0-4a61-908c-62903d78032e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:17.955680 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1169/2345] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:34:17.992365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:17.992964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:17.992964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:18.019752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] MFA Rules not processed for user `2892e755ce4743adbd131ebda92931d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:18.021024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:18.035770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1177/2346] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:34:18.038864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62d9e6-43c0-4a61-908c-62903d78032e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:18.061519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:18.064023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:18.065249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62d9e6-43c0-4a61-908c-62903d78032e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:18.072581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:18.073283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:18.073564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:18.073891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] PATH_INFO: `/v3/users/d040957e3ea04505bfa9e3b3d7c66f52` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:18.075108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d040957e3ea04505bfa9e3b3d7c66f52)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:18.078235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:18.092030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b5942-9e0e-4b57-8c86-10e482a9dda6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:18.102692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:18.104551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:18.105044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:18.105575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:18.137797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:18.141189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:18.152180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd040957e3ea04505bfa9e3b3d7c66f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.153290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1178/2347] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:34:18.164458 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd040957e3ea04505bfa9e3b3d7c66f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.201367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd040957e3ea04505bfa9e3b3d7c66f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.212470 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd040957e3ea04505bfa9e3b3d7c66f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.232021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd040957e3ea04505bfa9e3b3d7c66f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.239307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1170/2348] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:34:18 2022] DELETE /identity/v3/users/d040957e3ea04505bfa9e3b3d7c66f52 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:34:18.512983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:18.554634 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d17b4-d805-43d4-9401-198a85b69b88 service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:18.651341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:18.652083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:18.652367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:18.652636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] PATH_INFO: `/v3/projects/465f32cb4f724c18aad10fca114a1321` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:18.653871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=465f32cb4f724c18aad10fca114a1321)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:18.655971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:18.708869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:18.722169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465f32cb4f724c18aad10fca114a1321', 'request_id': 'req-35359845-a20b-4ce7-bad6-a243f2cc7d5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.728323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465f32cb4f724c18aad10fca114a1321', 'request_id': 'req-35359845-a20b-4ce7-bad6-a243f2cc7d5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.737403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465f32cb4f724c18aad10fca114a1321', 'request_id': 'req-35359845-a20b-4ce7-bad6-a243f2cc7d5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.750728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] The token cache is being invalidate because project 465f32cb4f724c18aad10fca114a1321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:34:18.751015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.751952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:18.753708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1179/2349] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:34:18 2022] DELETE /identity/v3/projects/465f32cb4f724c18aad10fca114a1321 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:34:22.792307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:22.793756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:22.852263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8501e-391f-4cf5-be81-7412ff226315 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:22.855699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:22.901747 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:22.902355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:22.902631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:22.902905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:22.904677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:22.968973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:22.970412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:22.981434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1171/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:25.637840 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:25.639608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:25.751871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35359845-a20b-4ce7-bad6-a243f2cc7d5a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:25.756988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:25.757649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:25.757925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:25.758197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:25.760593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:25.775977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:25.777786 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:25.793355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1180/2351] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:26.507382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:26.508967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:26.523516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297e68b-382f-48b9-9275-582ba2441677 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:26.528910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:26.529498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:26.529802 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:26.530085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] PATH_INFO: `/v3/users/51657a0e4865469999bdf8cd5102dad1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:26.531436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=51657a0e4865469999bdf8cd5102dad1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:26.534632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:26.587260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51657a0e4865469999bdf8cd5102dad1'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:26.587945 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51657a0e4865469999bdf8cd5102dad1'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:26.596293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51657a0e4865469999bdf8cd5102dad1'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:26.606444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51657a0e4865469999bdf8cd5102dad1'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:26.618986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-76b63bb8-4d97-47fa-a07c-f1c623052a2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51657a0e4865469999bdf8cd5102dad1'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:26.625122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1172/2352] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:34:26 2022] DELETE /identity/v3/users/51657a0e4865469999bdf8cd5102dad1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:34:26.893951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:26.905337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9ad4-1856-4be8-8226-231c24d2e267 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:26.977238 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:26.978048 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:26.978453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:26.978892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] PATH_INFO: `/v3/projects/736d8803a13e4dddb1ef32b7ab11eb1c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:26.980285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=736d8803a13e4dddb1ef32b7ab11eb1c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:26.982353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.031074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:27.032260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736d8803a13e4dddb1ef32b7ab11eb1c', 'request_id': 'req-2952ad68-ff20-4299-8956-232f6b2e7c2b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:27.042272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736d8803a13e4dddb1ef32b7ab11eb1c', 'request_id': 'req-2952ad68-ff20-4299-8956-232f6b2e7c2b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:27.050041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736d8803a13e4dddb1ef32b7ab11eb1c', 'request_id': 'req-2952ad68-ff20-4299-8956-232f6b2e7c2b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:27.062066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] The token cache is being invalidate because project 736d8803a13e4dddb1ef32b7ab11eb1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:34:27.062360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:27.062995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2952ad68-ff20-4299-8956-232f6b2e7c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:34:27.064580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1181/2353] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:34:26 2022] DELETE /identity/v3/projects/736d8803a13e4dddb1ef32b7ab11eb1c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:34:27.070589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.070589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.070930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.092040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:27.093247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.118001 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:27.157761 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1173/2354] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:27.166225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.166514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.166791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.186533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:27.188060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.196341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1182/2355] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:27.202014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.216044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc500a74-943a-4882-b2dc-35850dc312f9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.223043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.226229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.226229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.226229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.226229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:34:27.226229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.227608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.239560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1174/2356] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:34:27.244106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.254040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d16d0-cb31-4309-a9a5-e141bfb04a0a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.254651 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.256399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.258750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.259361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.259641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.259942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.260637 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.262540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.298120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1183/2357] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:34:27 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:34:27.302636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.349925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaf77a-0b6a-4236-bfb2-648d695e5c64 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.366919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dfc95-0517-476e-bdaa-92be2b40e9cc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.373794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.374445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.374724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.374998 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.375660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.377646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.400357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1184/2358] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:34:27 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:34:27.404367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.414866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153ad7-4a3d-4113-a395-34be04ea5fac admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.419620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.420341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.420500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.420689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.420888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.421075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.421361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.421603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.421731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.423206 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.423464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.433737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1185/2359] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:34:27.437920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.448531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09362f-4480-4830-ba9e-3c3da5cfac1f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.453382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.453999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.454283 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.454559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.455293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.456967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.471930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1186/2360] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:34:27.474722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.479535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:27.490661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5c10-9419-4c33-8aa4-b6c32873f209 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.492042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.493749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd15a9-5b22-4a8c-8780-e98998352120 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.495537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.496279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.496555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.496829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] PATH_INFO: `/v3/projects/33ea04cddf084bc9b4aab1e7c477abbf/users/691d2cd293b244398bc9a3a98f1ad928/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.503804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1175/2361] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:27.514511 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ea04cddf084bc9b4aab1e7c477abbf, user_id=691d2cd293b244398bc9a3a98f1ad928, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.517524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.532959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1187/2362] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:34:27 2022] PUT /identity/v3/projects/33ea04cddf084bc9b4aab1e7c477abbf/users/691d2cd293b244398bc9a3a98f1ad928/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:34:27.538751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41a70-b39c-4936-9e8f-d7927a3a8c6e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.539322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41a70-b39c-4936-9e8f-d7927a3a8c6e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.539322 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41a70-b39c-4936-9e8f-d7927a3a8c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.561682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-02b41a70-b39c-4936-9e8f-d7927a3a8c6e None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:27.563185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02b41a70-b39c-4936-9e8f-d7927a3a8c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.634849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1176/2363] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:27.645516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:27.648365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:27.725479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d433-a58a-47c9-95cb-f54244bc5ad0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:27.730244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:27.731342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:27.731750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:27.732175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:27.739753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:27.750254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:27.752807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:27.768085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1188/2364] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:34:33.166919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1f791-7031-44e3-8b1a-91e0eab09052 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:33.167472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1f791-7031-44e3-8b1a-91e0eab09052 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:33.167472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1f791-7031-44e3-8b1a-91e0eab09052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:33.195968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e7a1f791-7031-44e3-8b1a-91e0eab09052 None None] MFA Rules not processed for user `691d2cd293b244398bc9a3a98f1ad928`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:33.197188 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e7a1f791-7031-44e3-8b1a-91e0eab09052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:33.262258 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1177/2365] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:33.276329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:33.277921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:33.352688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb727-02f4-4710-99cf-cbaa45e71462 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:33.358316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:33.358933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:33.359212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:33.359484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:33.368193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:33.369738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751da21d-0d4e-43ea-81a6-0910afe181d2 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:33.378531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1189/2366] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:34:34.725573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:34.726060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:34.726153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:34.747014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] MFA Rules not processed for user `691d2cd293b244398bc9a3a98f1ad928`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:34.748271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:34.758832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1178/2367] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:43.374548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=37 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:44.741298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=66 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:48.027965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68f94c-4fd3-4fcd-8849-aa7906ce0191 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:48.028741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68f94c-4fd3-4fcd-8849-aa7906ce0191 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:48.028925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68f94c-4fd3-4fcd-8849-aa7906ce0191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:48.049346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bf68f94c-4fd3-4fcd-8849-aa7906ce0191 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:34:48.050590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bf68f94c-4fd3-4fcd-8849-aa7906ce0191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:48.111081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1190/2368] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:34:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:34:48.162726 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:34:48.164295 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:34:48.247595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a30ca6-e2cc-423f-8849-a5051b7dfbe7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:34:48.252979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:34:48.253621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:34:48.253903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:34:48.254179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:34:48.263608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:34:48.265246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20b88-0228-4d58-acf9-76c16a90a751 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:34:48.275094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1179/2369] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:34:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 15:34:58.104049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=25 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:34:58.271682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=32 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:01.655562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.656211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.656425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:01.658057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1191/2370] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:35:01.665646 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7cba-594d-4d28-86a9-8cbbee497ec2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.666213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7cba-594d-4d28-86a9-8cbbee497ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.666573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7cba-594d-4d28-86a9-8cbbee497ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:01.691207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-82dd7cba-594d-4d28-86a9-8cbbee497ec2 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:01.692712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-82dd7cba-594d-4d28-86a9-8cbbee497ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:01.704593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1180/2371] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:01.720392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:01.722643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:01.806880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f3532-2da1-4d8f-bf11-2f301c23c403 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:01.811178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:01.812165 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.812470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.812961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:01.824699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:01.826146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be672-6709-4616-8d01-99d2436e8840 service glance] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:01.839447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1192/2372] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:35:01.923682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.924014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.924217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:01.925595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1181/2373] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:35:01.930266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea3d6f-9abd-40a2-9d5c-8d78f07c57ce None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.930623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea3d6f-9abd-40a2-9d5c-8d78f07c57ce None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.930946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea3d6f-9abd-40a2-9d5c-8d78f07c57ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:01.951033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-82ea3d6f-9abd-40a2-9d5c-8d78f07c57ce None None] MFA Rules not processed for user `94eb8ac7bd934c318f0e52833928c589`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:01.952322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82ea3d6f-9abd-40a2-9d5c-8d78f07c57ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:01.960358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1193/2374] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:01.975405 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:01.977356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:01.992653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1a86-b48b-4330-8e1d-1631f10edf67 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:01.997088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:01.997863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:01.998279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:01.998569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:02.008060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:02.009564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999e238-8693-459e-b249-e442efab7124 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:02.018316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1182/2375] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:35:02.107518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd4cf4-2d92-423a-a58e-ccbf126a60e6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:02.107874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd4cf4-2d92-423a-a58e-ccbf126a60e6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:02.108196 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd4cf4-2d92-423a-a58e-ccbf126a60e6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:02.109523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1194/2376] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 15:35:02.116370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa0d25-1a13-4ca0-8021-6e7414d97a02 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:02.116901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa0d25-1a13-4ca0-8021-6e7414d97a02 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:02.117483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa0d25-1a13-4ca0-8021-6e7414d97a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:02.140366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85aa0d25-1a13-4ca0-8021-6e7414d97a02 None None] MFA Rules not processed for user `fe58a99358bc42aebeb126e5ef2cf21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:02.141690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85aa0d25-1a13-4ca0-8021-6e7414d97a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:02.152363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1183/2377] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:35:02.163336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd4cf4-2d92-423a-a58e-ccbf126a60e6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:02.174697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd4cf4-2d92-423a-a58e-ccbf126a60e6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:02.178198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:02.178798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:02.179075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:02.179345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:02.189746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:02.191169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:02.202964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1195/2378] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:35:03.972090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83bbfc-4d30-4dcc-8f5a-5e2ead718e05 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:03.972572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83bbfc-4d30-4dcc-8f5a-5e2ead718e05 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:03.972677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83bbfc-4d30-4dcc-8f5a-5e2ead718e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:03.992235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-af83bbfc-4d30-4dcc-8f5a-5e2ead718e05 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:03.993425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-af83bbfc-4d30-4dcc-8f5a-5e2ead718e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:04.000931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1184/2379] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:35:04.016515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:04.029303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580acbd1-fd74-4fc2-acdd-7c2981a75f41 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:04.034308 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:04.034910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:04.035195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:04.035526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:04.045062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:04.046704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:04.056633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1196/2380] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:35:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:35:13.986692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=15 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:14.052337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=41 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:19.345637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9310-892d-4db7-85f6-8975bafcb885 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:19.348096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9310-892d-4db7-85f6-8975bafcb885 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:19.348096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9310-892d-4db7-85f6-8975bafcb885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:19.366889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fcce9310-892d-4db7-85f6-8975bafcb885 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:19.368189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fcce9310-892d-4db7-85f6-8975bafcb885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:19.375498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1185/2381] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:19.389007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:19.402350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381327-2919-4a50-8fcb-76ccbe133c23 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:19.410505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:19.411533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:19.412033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:19.412530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:19.423401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:19.425043 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:19.434201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1197/2382] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:35:29.361388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:29.430616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:40.905943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:40.906544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:40.906776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:40.931088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] MFA Rules not processed for user `691d2cd293b244398bc9a3a98f1ad928`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:40.932406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:40.941291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1186/2383] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:35:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:41.053934 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:41.056966 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:41.182222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31a67a-ab6e-4581-ac18-454d6a7aa0a0 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:41.187964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:41.188622 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:41.188902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:41.189176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:41.191062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:41.321960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:41.324575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e089b-c76d-4f9a-8f10-1ca602a2057d service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:41.372038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1198/2384] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:35:41.426340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:41.439179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846edb0-8fde-4d92-ab5e-9b827fc83819 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:41.443416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:41.444056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:41.444337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:41.444606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:41.446491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:41.459564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:41.463570 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:41.474195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1187/2385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:35:49.557651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:49.558598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:49.558867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:49.581940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] MFA Rules not processed for user `d737eac8ae0343df9ace919d75596800`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:49.583218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:49.590662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1199/2386] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:51.469345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=9 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:52.260058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:52.280419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d59e1e-532a-498e-941d-a290a1bbaac6 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:52.293015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:52.293859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:52.294477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:52.294972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:52.298053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:52.324704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:52.326523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:52.348866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1188/2387] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:35:58.571850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:58.574248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:58.669951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:58.672378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:58.691411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579a95c-ae5f-4a77-b4d4-ab4155dc34a3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:58.696566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa3e-f210-4fd8-bb9c-97d85a79ca69 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:58.698662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:58.699574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:58.700410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:58.701358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:58.709527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:58.711319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:58.711766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:58.712231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] PATH_INFO: `/v3/users/a2b027a2c8dd4459b1309faab28f3d4b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:58.714017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b027a2c8dd4459b1309faab28f3d4b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:58.717466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:58.719424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:58.752790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:58.756763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:58.775386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1200/2388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:35:58.810678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b027a2c8dd4459b1309faab28f3d4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:58.811453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b027a2c8dd4459b1309faab28f3d4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:58.826588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b027a2c8dd4459b1309faab28f3d4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:58.838889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b027a2c8dd4459b1309faab28f3d4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:58.855214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0acc5c03-0168-4c0d-9b35-a0ade87ae10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b027a2c8dd4459b1309faab28f3d4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:58.861776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1189/2389] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:35:58 2022] DELETE /identity/v3/users/a2b027a2c8dd4459b1309faab28f3d4b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:35:59.049422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.068326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45897-cf4e-4bfc-9b7c-0e30d9e6a26d service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.119249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:59.150387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.151057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.151344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.152110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] PATH_INFO: `/v3/projects/fddbf2261d1f42d0a41aa921197fc181` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.152897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddbf2261d1f42d0a41aa921197fc181)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.154851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.215548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddbf2261d1f42d0a41aa921197fc181', 'request_id': 'req-e4bebb99-6617-4c51-bd0c-ada758d29613'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:59.223192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddbf2261d1f42d0a41aa921197fc181', 'request_id': 'req-e4bebb99-6617-4c51-bd0c-ada758d29613'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:59.232550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddbf2261d1f42d0a41aa921197fc181', 'request_id': 'req-e4bebb99-6617-4c51-bd0c-ada758d29613'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:59.252903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] The token cache is being invalidate because project fddbf2261d1f42d0a41aa921197fc181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:35:59.253366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:59.254330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4bebb99-6617-4c51-bd0c-ada758d29613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:35:59.260788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1201/2390] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:35:59 2022] DELETE /identity/v3/projects/fddbf2261d1f42d0a41aa921197fc181 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:35:59.265715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.266088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.266430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.293058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:59.294641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:59.362724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1190/2391] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:59.370858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93642-5c12-425d-af30-013d67033671 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.371143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93642-5c12-425d-af30-013d67033671 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.371421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93642-5c12-425d-af30-013d67033671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.391988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-82c93642-5c12-425d-af30-013d67033671 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:59.393418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82c93642-5c12-425d-af30-013d67033671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:59.402644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1202/2392] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:59.407142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.418668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b02b11-2a72-4dc1-93b1-ba115f72b62b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.424028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.424922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.425203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.425472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.426181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:35:59.426478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.428115 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.441804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1191/2393] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:35:59.446310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93642-5c12-425d-af30-013d67033671 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.457062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93642-5c12-425d-af30-013d67033671 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.462394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.463020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.463299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.463684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.464344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.466190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.498225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1203/2394] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:35:59 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:35:59.501638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.512898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9990b8-5ec0-4cf7-abb5-ddd6ad5f6163 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.560674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:35:59.577549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.578098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.578376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.578660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.579344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.581903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.604916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1192/2395] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:35:59 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:35:59.609069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.620249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1ac1-d347-41ed-9ac6-773f1cf7a6de admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.625395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.626064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.626345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.626619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.627331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.629035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.640703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1204/2396] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:35:59.645243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.655720 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef258-2632-43d3-b21f-bef293434632 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.661581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.662202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.662483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.662758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.663451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.665136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247427d-4591-4b5e-9fe7-dc17c3315c28 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.676239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1193/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:35:59.680525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.691418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3d49-3a10-4ac6-8082-002e4a9bb854 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.696202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.696871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.697151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.697425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] PATH_INFO: `/v3/projects/3360c121308844848a03ff34c6fd73f0/users/b9c1adb6cda8489dbfc30e80ee8ca8b6/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.712765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3360c121308844848a03ff34c6fd73f0, user_id=b9c1adb6cda8489dbfc30e80ee8ca8b6, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.715434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.728753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1205/2398] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:35:59 2022] PUT /identity/v3/projects/3360c121308844848a03ff34c6fd73f0/users/b9c1adb6cda8489dbfc30e80ee8ca8b6/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:35:59.735751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8614-b90f-4222-8e8b-6a769092829a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.736095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8614-b90f-4222-8e8b-6a769092829a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.736377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8614-b90f-4222-8e8b-6a769092829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.756014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c4bd8614-b90f-4222-8e8b-6a769092829a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:35:59.757238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c4bd8614-b90f-4222-8e8b-6a769092829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:59.830185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1194/2399] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:35:59.840656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:35:59.842491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:35:59.913484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9498f-c4ec-4491-ac68-bfa6c7486063 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:35:59.917367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:35:59.917988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:35:59.918269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:35:59.918545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:35:59.927810 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:35:59.929275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:35:59.939113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1206/2400] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:00.716928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:00.717508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:00.717508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:00.757009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] MFA Rules not processed for user `b9c1adb6cda8489dbfc30e80ee8ca8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:00.758847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:00.842831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1195/2401] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:00.849110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:00.861007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee729fff-fa2c-43e2-9b6b-ccd582d1b10b service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:00.866885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:00.867815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:00.868200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:00.868706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:00.869507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:00.871707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:00.876901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:00.908272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1207/2402] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:00 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:00.912417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:00.925409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae756c0-221f-4d30-b902-f9b612c8cb64 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:00.986732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:01.000841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:01.001907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:01.002408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:01.003459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:01.004636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:01.008464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:01.051881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1196/2403] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:00 2022] POST /identity/v3/users => generated 308 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:01.057842 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:01.078758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf8c29-ca72-4997-a66c-6494ce1242cf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:01.085752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:01.086417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:01.086695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:01.086964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:01.087670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:01.089659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:01.105005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1208/2404] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:01.106642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:01.126358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0550d1-97f2-48df-86a1-0bc0df31ae68 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:01.135899 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:01.137320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:01.137809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:01.138227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] PATH_INFO: `/v3/projects/70673091cbd043ce961dec96d986d5d7/users/26a769c482e34cd29eb11d84222a9905/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:01.156953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70673091cbd043ce961dec96d986d5d7, user_id=26a769c482e34cd29eb11d84222a9905, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:01.159776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:01.174293 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1197/2405] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:01 2022] PUT /identity/v3/projects/70673091cbd043ce961dec96d986d5d7/users/26a769c482e34cd29eb11d84222a9905/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:01.178685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:01.190095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440aeb0-ef60-4016-b9cb-412f38a30e07 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:01.258977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:01.259688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:01.260086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:01.260443 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:01.261161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:01.262841 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baeb73e-3bdc-4796-be69-8e3d39f63fcb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:01.278076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1209/2406] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:01.279079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:01.290389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdcb30-5004-4fe4-91db-3285af77e3d3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:01.297179 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:01.298031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:01.298318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:01.298597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] PATH_INFO: `/v3/projects/70673091cbd043ce961dec96d986d5d7/users/26a769c482e34cd29eb11d84222a9905/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:01.300776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70673091cbd043ce961dec96d986d5d7, user_id=26a769c482e34cd29eb11d84222a9905, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:01.303494 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:01.328594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1198/2407] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:01 2022] PUT /identity/v3/projects/70673091cbd043ce961dec96d986d5d7/users/26a769c482e34cd29eb11d84222a9905/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:02.066846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:02.067341 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:02.067457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:02.098255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] MFA Rules not processed for user `26a769c482e34cd29eb11d84222a9905`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:02.099474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:02.190656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1210/2408] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:03.372728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:03.439763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8155f6-8503-4463-8636-4321be65a57f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:03.444422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:03.445071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:03.445350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:03.445627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] PATH_INFO: `/v3/users/b9c1adb6cda8489dbfc30e80ee8ca8b6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:03.446794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c1adb6cda8489dbfc30e80ee8ca8b6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:03.448760 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:03.501502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c1adb6cda8489dbfc30e80ee8ca8b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:03.502178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c1adb6cda8489dbfc30e80ee8ca8b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:03.511340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c1adb6cda8489dbfc30e80ee8ca8b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:03.518895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c1adb6cda8489dbfc30e80ee8ca8b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:03.533476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c1adb6cda8489dbfc30e80ee8ca8b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:03.540561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1199/2409] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:03 2022] DELETE /identity/v3/users/b9c1adb6cda8489dbfc30e80ee8ca8b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:03.835619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:03.848362 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e95b45-cdf5-49c2-985d-d392f4d8b596 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:03.915144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:03.923449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:03.924147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:03.925190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:03.925653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] PATH_INFO: `/v3/projects/3360c121308844848a03ff34c6fd73f0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:03.927020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3360c121308844848a03ff34c6fd73f0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:03.929193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:03.994961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3360c121308844848a03ff34c6fd73f0', 'request_id': 'req-447bf206-8ecd-4307-9a87-e6172833db5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.007697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3360c121308844848a03ff34c6fd73f0', 'request_id': 'req-447bf206-8ecd-4307-9a87-e6172833db5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.018793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3360c121308844848a03ff34c6fd73f0', 'request_id': 'req-447bf206-8ecd-4307-9a87-e6172833db5a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.034636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] The token cache is being invalidate because project 3360c121308844848a03ff34c6fd73f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:04.034920 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.035551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.037149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1211/2410] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:03 2022] DELETE /identity/v3/projects/3360c121308844848a03ff34c6fd73f0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:04.042302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:04.043961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:04.060551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbf138-9c34-4d8e-b4c0-826bce80acc1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:04.102782 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:04.157528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:04.158180 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.158463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.158735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] PATH_INFO: `/v3/users/26a769c482e34cd29eb11d84222a9905` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.160443 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a769c482e34cd29eb11d84222a9905)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:04.162364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:04.232714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a769c482e34cd29eb11d84222a9905'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.233650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a769c482e34cd29eb11d84222a9905'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.243942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a769c482e34cd29eb11d84222a9905'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.261220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a769c482e34cd29eb11d84222a9905'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.275200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-16236dfd-5212-4b33-9d6f-17d7fece3c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a769c482e34cd29eb11d84222a9905'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.286248 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1200/2411] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:04 2022] DELETE /identity/v3/users/26a769c482e34cd29eb11d84222a9905 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:04.475272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:04.477541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:04.494550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bf206-8ecd-4307-9a87-e6172833db5a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:04.574561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:04.575377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.575674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.576003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] PATH_INFO: `/v3/projects/70673091cbd043ce961dec96d986d5d7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.577295 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70673091cbd043ce961dec96d986d5d7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:04.579489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:04.672315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70673091cbd043ce961dec96d986d5d7', 'request_id': 'req-50e137ef-b9a5-4ca1-9338-26b1c1278b91'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.683558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70673091cbd043ce961dec96d986d5d7', 'request_id': 'req-50e137ef-b9a5-4ca1-9338-26b1c1278b91'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.698388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70673091cbd043ce961dec96d986d5d7', 'request_id': 'req-50e137ef-b9a5-4ca1-9338-26b1c1278b91'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.715394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] The token cache is being invalidate because project 70673091cbd043ce961dec96d986d5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:04.715692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.716690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-50e137ef-b9a5-4ca1-9338-26b1c1278b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:04.719022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1212/2412] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:04 2022] DELETE /identity/v3/projects/70673091cbd043ce961dec96d986d5d7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:04.724577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.725805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.726396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.760154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:04.761496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:04.852691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1201/2413] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:04.862790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.863855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.864372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.888706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:04.890428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:04.899769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1213/2414] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:04.904016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:04.914964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca348561-a60e-42ef-83b4-e36e2e94f334 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:04.920369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:04.921068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.921350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.921627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.922341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:04.922650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:04.924310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:04.940650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1202/2415] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:04.945883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:04.959667 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16a289-1fda-45d0-9113-da9315275a00 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:04.972019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:04.972660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:04.972944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:04.973221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:04.973900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:04.976076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:04.981925 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:05.015992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1214/2416] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:04 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:05.020847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.041016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff14641-2803-4349-8871-fb046acbe515 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.089368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:05.138348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.140251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.140251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.140251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.140636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.142976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.180390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1203/2417] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:05 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:05.185108 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.202410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bfe8d-71d6-41a6-92b2-0b164b5b2c90 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.209149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.210508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.210955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.211515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.212632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.215045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.227864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1215/2418] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:05.231554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.244447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d778b-1fd1-48da-a9a5-a84357a9cdc7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.250514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.251560 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.251873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.252158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.252888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.254737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec696a51-c694-4aa8-914c-ba7e1a0b7ed0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.269129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1204/2419] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:05.273001 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.283765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b5525-33e3-4311-8c3c-00c5ae6fc031 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.289576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.290701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.290701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.290910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] PATH_INFO: `/v3/projects/33df457d13af428db2580a5ca212b528/users/6d0f7ea712ad49a2aa994eb27d7aad12/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.310984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33df457d13af428db2580a5ca212b528, user_id=6d0f7ea712ad49a2aa994eb27d7aad12, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.313730 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.329303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1216/2420] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:05 2022] PUT /identity/v3/projects/33df457d13af428db2580a5ca212b528/users/6d0f7ea712ad49a2aa994eb27d7aad12/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:05.340594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.341234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.341762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.378413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] MFA Rules not processed for user `6d0f7ea712ad49a2aa994eb27d7aad12`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:05.380292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:05.470553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1205/2421] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:05.475781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.563628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eb87e-0004-4f43-853d-34662972a123 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.568880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.569510 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.569817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.570237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.570954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.572982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.601874 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1217/2422] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:05 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:05.606059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.618420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39170d70-e984-4d0f-9b27-59169c05fe38 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.680581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.681223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.681504 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.681783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.682656 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.684904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.710731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1206/2423] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:05 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:05.715112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.728754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434891f-f32f-45f7-bc9e-d353bfc8aca7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.733497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.734131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.734419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.734698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.735403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.737086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.760574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1218/2424] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:05.761849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.773110 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1df57-3bb8-445b-9993-a4951244efba admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.783299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.784149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.784499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.784843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] PATH_INFO: `/v3/projects/ad1f740c0b4a471986f169b71c158988/users/c9eb3c6f78b241c2b3e3e7143ce1a79b/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.801578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1f740c0b4a471986f169b71c158988, user_id=c9eb3c6f78b241c2b3e3e7143ce1a79b, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.804378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.825136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1207/2425] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:05 2022] PUT /identity/v3/projects/ad1f740c0b4a471986f169b71c158988/users/c9eb3c6f78b241c2b3e3e7143ce1a79b/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:05.829790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.843546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2367e-ccfb-4ee5-97f7-4818eb4027ea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.928843 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.929509 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.929745 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.930020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.930854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.932535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf00b-c0ad-4e70-9629-340fcae54842 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.945063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1219/2426] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:05.956181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:05.970308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a5ec-df47-415f-84b0-e3061b3400ae admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:05.975797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:05.976519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:05.976804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:05.977080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] PATH_INFO: `/v3/projects/ad1f740c0b4a471986f169b71c158988/users/c9eb3c6f78b241c2b3e3e7143ce1a79b/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:05.979135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1f740c0b4a471986f169b71c158988, user_id=c9eb3c6f78b241c2b3e3e7143ce1a79b, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:05.982170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:05.984781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:05.996375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1208/2427] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:05 2022] PUT /identity/v3/projects/ad1f740c0b4a471986f169b71c158988/users/c9eb3c6f78b241c2b3e3e7143ce1a79b/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:06.009557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:06.009844 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:06.010125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:06.040570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] MFA Rules not processed for user `c9eb3c6f78b241c2b3e3e7143ce1a79b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:06.041808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:06.052906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:06.166623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1220/2428] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:06.182391 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:06.185269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:06.273715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f782364-92b7-40b9-9238-bb5182865c76 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:06.278420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:06.279052 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:06.279455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:06.279608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:06.295416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:06.296964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:06.307767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1209/2429] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:06.364782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:06.366383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:06.380058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86189716-3fc4-499b-97af-07cbf7e2cd6e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:06.384158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:06.384985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:06.385309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:06.385658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:06.471865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:06.474619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:06.496259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1221/2430] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:09.520820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:09.522384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:09.587607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2690d19-6102-468a-a1d6-470efbec698b service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:09.591366 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:09.592120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:09.592406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:09.592681 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:09.594546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:09.685534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:09.687079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee2a87-ac3c-40d2-a0ce-349a5acda823 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:09.698397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1210/2431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:10.292852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:10.306396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac002-d5bf-46b3-bdbb-ef9de1acf0bb service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:10.314658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:10.315521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:10.316154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:10.316660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] PATH_INFO: `/v3/users/6d0f7ea712ad49a2aa994eb27d7aad12` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:10.318058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0f7ea712ad49a2aa994eb27d7aad12)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:10.321328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:10.375465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0f7ea712ad49a2aa994eb27d7aad12'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:10.385074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0f7ea712ad49a2aa994eb27d7aad12'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:10.398240 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0f7ea712ad49a2aa994eb27d7aad12'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:10.405349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0f7ea712ad49a2aa994eb27d7aad12'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:10.406187 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0f7ea712ad49a2aa994eb27d7aad12'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:10.413892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1222/2432] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:10 2022] DELETE /identity/v3/users/6d0f7ea712ad49a2aa994eb27d7aad12 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:10.418403 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:10.418770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:10.419160 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:10.441454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:10.442695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:10.488327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:10.509825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1211/2433] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:10.520512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:10.522244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:10.544555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:10.590479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7212-4a09-44c2-8965-ce2550f928d5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:10.594250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:10.594885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:10.595163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:10.595438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:10.607894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:10.610021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:10.623698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1223/2434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:11.291914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:11.293521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:11.306886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f6c04-1770-4359-a84f-3d46b5b5c286 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:11.317053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:11.317876 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:11.318230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:11.318619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] PATH_INFO: `/v3/projects/33df457d13af428db2580a5ca212b528` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:11.320068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33df457d13af428db2580a5ca212b528)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:11.329905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:11.424764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33df457d13af428db2580a5ca212b528', 'request_id': 'req-20e4cfbd-e961-4437-a13c-3b9e53988f0d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.433386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33df457d13af428db2580a5ca212b528', 'request_id': 'req-20e4cfbd-e961-4437-a13c-3b9e53988f0d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.443571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33df457d13af428db2580a5ca212b528', 'request_id': 'req-20e4cfbd-e961-4437-a13c-3b9e53988f0d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.460240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] The token cache is being invalidate because project 33df457d13af428db2580a5ca212b528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:11.460667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.461395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.463926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1212/2435] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:11 2022] DELETE /identity/v3/projects/33df457d13af428db2580a5ca212b528 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:11.467670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:11.481778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36039e87-c305-4d35-b53e-96df5fd4e358 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:11.563383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:11.564505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:11.564855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:11.565801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] PATH_INFO: `/v3/users/c9eb3c6f78b241c2b3e3e7143ce1a79b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:11.566462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9eb3c6f78b241c2b3e3e7143ce1a79b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:11.568607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:11.638781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9eb3c6f78b241c2b3e3e7143ce1a79b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.650699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9eb3c6f78b241c2b3e3e7143ce1a79b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.672817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9eb3c6f78b241c2b3e3e7143ce1a79b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.685769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9eb3c6f78b241c2b3e3e7143ce1a79b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.686799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9eb3c6f78b241c2b3e3e7143ce1a79b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:11.696676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1224/2436] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:11 2022] DELETE /identity/v3/users/c9eb3c6f78b241c2b3e3e7143ce1a79b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:11.809785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:11.811241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:11.957385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4cfbd-e961-4437-a13c-3b9e53988f0d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.043521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.044678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.045177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.045569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.047725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:12.111114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:12.130276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.131891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:12.180719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.183359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2dca-0971-4483-8790-284dc046f974 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.198060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1213/2437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:12.202971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:12.232639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0d04-0d91-48b2-b650-97b2b955d2e2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.239593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.240984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.241911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.241911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] PATH_INFO: `/v3/projects/ad1f740c0b4a471986f169b71c158988` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.243661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1f740c0b4a471986f169b71c158988)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.245457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.329530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1f740c0b4a471986f169b71c158988', 'request_id': 'req-615a1e40-d560-47b4-9504-e386417513e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:12.341816 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1f740c0b4a471986f169b71c158988', 'request_id': 'req-615a1e40-d560-47b4-9504-e386417513e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:12.362956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1f740c0b4a471986f169b71c158988', 'request_id': 'req-615a1e40-d560-47b4-9504-e386417513e0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:12.380818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] The token cache is being invalidate because project ad1f740c0b4a471986f169b71c158988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:12.381406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:12.382222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-615a1e40-d560-47b4-9504-e386417513e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:12.384261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1225/2438] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:12 2022] DELETE /identity/v3/projects/ad1f740c0b4a471986f169b71c158988 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:12.389203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.390456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.390754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.412949 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:12.414209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:12.513749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1214/2439] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:12.528705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.529301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.529629 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.564261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:12.564261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:12.574024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1226/2440] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:12.578526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.592266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bf75a-6a5a-4546-94b4-d82fd7f34059 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.598607 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.599317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.599674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.600117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.601186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:12.601678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.603889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.617388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1215/2441] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:12.621575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.632771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e9a88-3243-44d4-8524-f5a95b78cff6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.642190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.642859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.643211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.643723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.644884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.647031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.681536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1227/2442] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:12 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:12.687660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.709100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d818-6f3b-4782-9045-af7365027d42 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.779287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.779950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.780243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.780614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.781532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.784951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.807922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1216/2443] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:12 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:12.812328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.823695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecdbf9-a99d-4d3e-9168-6017084a3737 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.829809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.831201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.831485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.831762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.832504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.834154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.845200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1228/2444] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:12.857328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.870615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2037-b609-4daf-b595-3f1f1b856dc8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.876700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.877970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.877970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.877970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.881678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.881678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37268769-12c1-45ba-9954-5387f24eb4bd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.903204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1217/2445] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:12.908045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:12.923276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2842c-c136-4edf-8797-87b54bcd2f18 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:12.936004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:12.936701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.936987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.937273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] PATH_INFO: `/v3/projects/499e0f15738747008e4627eb5173a991/users/e5f7128822ef4d7e8fa6d7e8ac40072f/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:12.964021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499e0f15738747008e4627eb5173a991, user_id=e5f7128822ef4d7e8fa6d7e8ac40072f, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:12.964021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:12.985818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1229/2446] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:12 2022] PUT /identity/v3/projects/499e0f15738747008e4627eb5173a991/users/e5f7128822ef4d7e8fa6d7e8ac40072f/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:12.998353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:12.998969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:12.999242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.041316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] MFA Rules not processed for user `e5f7128822ef4d7e8fa6d7e8ac40072f`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:13.043237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:13.095025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:13.132565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1218/2447] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:13.137293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.211602 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897b9e-30b7-47f2-8ad5-c130fe574fe2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.216706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.217339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.217617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.217891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.218625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.220594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.234961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:13.259727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1230/2448] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:13 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:13.264210 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.278558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe814b6-c0e6-46fd-833c-7d97bd411105 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.363880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.365053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.365338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.365597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.366479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.368262 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.408195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1219/2449] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:13 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:13.409382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.430328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061629fe-af71-4db8-b7d0-4c4ae2ae7128 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.445146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.447803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.448833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.450058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.451069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.453242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.465612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1231/2450] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:13.469908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.485752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3becdf-46af-491b-8fcf-fd3c3f02cad3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.493597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.494541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.494989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.495445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] PATH_INFO: `/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb/users/0a72e1bd69b041fb912353db6c2de079/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.532823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc7e50586884fd7b68d8fe2a867b3bb, user_id=0a72e1bd69b041fb912353db6c2de079, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.536030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.564923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1220/2451] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:13 2022] PUT /identity/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb/users/0a72e1bd69b041fb912353db6c2de079/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:13.566052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.578394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b06c9-2260-429c-ad3d-e57e8f3e1d7f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.680577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.681372 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.681664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.682361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.683650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.686034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff8a3d-ac1f-4269-894f-0e113a123c3b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.702621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.703072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1232/2452] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:13.715308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1559-2692-4270-b0a8-57af89455f11 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:13.721592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:13.722266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.722508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.722788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] PATH_INFO: `/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb/users/0a72e1bd69b041fb912353db6c2de079/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.725217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc7e50586884fd7b68d8fe2a867b3bb, user_id=0a72e1bd69b041fb912353db6c2de079, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:13.728357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:13.754555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:13.754847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:13.755132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:13.758086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1221/2453] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:13 2022] PUT /identity/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb/users/0a72e1bd69b041fb912353db6c2de079/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:13.790641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] MFA Rules not processed for user `0a72e1bd69b041fb912353db6c2de079`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:13.792652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:13.974957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1233/2454] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:13.988150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:13.990274 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:14.107676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225129ce-7ad4-4768-90e3-663ca1fc5944 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:14.112550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:14.113417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.114519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.114864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.127717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:14.129199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:14.143758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1222/2455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:14.230734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:14.334571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e57ecf-a64d-4a76-be49-77c8d495fc07 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:14.339801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:14.340590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.341006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.345547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] PATH_INFO: `/v3/users/e5f7128822ef4d7e8fa6d7e8ac40072f` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.346529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:14.347576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:14.348752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f7128822ef4d7e8fa6d7e8ac40072f)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:14.350765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:14.355476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:14.363388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0848-51e7-4aa1-bf97-7ee3b15921a4 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:14.370499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:14.371354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.371753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.372327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] PATH_INFO: `/v3/users/7a964a003f8e4f1da603a13eb7f534ca` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.373436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a964a003f8e4f1da603a13eb7f534ca)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:14.375435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:14.400807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:14.418525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f7128822ef4d7e8fa6d7e8ac40072f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.433176 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f7128822ef4d7e8fa6d7e8ac40072f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.437376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a964a003f8e4f1da603a13eb7f534ca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.438383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a964a003f8e4f1da603a13eb7f534ca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.447408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a964a003f8e4f1da603a13eb7f534ca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.451909 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f7128822ef4d7e8fa6d7e8ac40072f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.456043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a964a003f8e4f1da603a13eb7f534ca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.462886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f7128822ef4d7e8fa6d7e8ac40072f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.464445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b9719715-7ad4-4019-ad8b-462c6849a289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f7128822ef4d7e8fa6d7e8ac40072f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.469992 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a964a003f8e4f1da603a13eb7f534ca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:14.471384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1234/2456] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:14 2022] DELETE /identity/v3/users/e5f7128822ef4d7e8fa6d7e8ac40072f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:14.475858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.476247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.476590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.478231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1223/2457] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:14 2022] DELETE /identity/v3/users/7a964a003f8e4f1da603a13eb7f534ca => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:14.501612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:14.502929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:14.581534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1235/2458] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:14.588778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:14.590473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:14.655339 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096c319-e2be-491a-8474-4e6ff7e33241 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:14.659600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:14.660257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.660540 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.660816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.670682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:14.672270 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e799ab-9b95-4dce-ad91-1132412efbb5 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:14.685097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1224/2459] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:14.924654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:14.926464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:14.945703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67187550-eda4-41fa-8b1a-bcc7cd1c42a8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:14.952011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:14.952847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:14.953129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:14.953469 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] PATH_INFO: `/v3/projects/ddfdba8b75e643b485b2237ae070a1b2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:14.954903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddfdba8b75e643b485b2237ae070a1b2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:14.957755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.031727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddfdba8b75e643b485b2237ae070a1b2', 'request_id': 'req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.043316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddfdba8b75e643b485b2237ae070a1b2', 'request_id': 'req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.062788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddfdba8b75e643b485b2237ae070a1b2', 'request_id': 'req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.090130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] The token cache is being invalidate because project ddfdba8b75e643b485b2237ae070a1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:15.090779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.092202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.096589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1236/2460] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:14 2022] DELETE /identity/v3/projects/ddfdba8b75e643b485b2237ae070a1b2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:15.103905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2512f-e478-4f6a-9be6-a79c76353059 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.104153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2512f-e478-4f6a-9be6-a79c76353059 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.104367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2512f-e478-4f6a-9be6-a79c76353059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.131762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f4d2512f-e478-4f6a-9be6-a79c76353059 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:15.137082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f4d2512f-e478-4f6a-9be6-a79c76353059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.152886 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.154537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.163728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:15.262398 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1225/2461] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:15.269508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d20ee-af3a-4ed3-8fe1-a705fc2a8634 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.275643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.275985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.276150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.276620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.276753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.276951 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.277323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] PATH_INFO: `/v3/projects/499e0f15738747008e4627eb5173a991` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.278473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=499e0f15738747008e4627eb5173a991)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.280453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.302955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:15.304345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.314731 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1226/2462] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:15.319712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.334250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a01f0-1f31-4d1e-926b-d347108faf3c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.363006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499e0f15738747008e4627eb5173a991', 'request_id': 'req-7eed2750-ee30-495f-b082-e35b58b31a8f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.370708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499e0f15738747008e4627eb5173a991', 'request_id': 'req-7eed2750-ee30-495f-b082-e35b58b31a8f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.383332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499e0f15738747008e4627eb5173a991', 'request_id': 'req-7eed2750-ee30-495f-b082-e35b58b31a8f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.402582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] The token cache is being invalidate because project 499e0f15738747008e4627eb5173a991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:15.402878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.403654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.405536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1237/2463] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:15 2022] DELETE /identity/v3/projects/499e0f15738747008e4627eb5173a991 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:15.409244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:15.410942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.414605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.442122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.442766 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.443038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.443313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.444062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:15.444367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.446831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.464485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1227/2464] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:15.470950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.475125 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.497930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f38f-63a5-4a19-9679-4ec4fcef5192 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.504303 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.505497 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.506297 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.506850 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.507783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.510098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.512917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed2750-ee30-495f-b082-e35b58b31a8f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.518946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.519775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.520100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.520385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] PATH_INFO: `/v3/users/0a72e1bd69b041fb912353db6c2de079` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.521933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a72e1bd69b041fb912353db6c2de079)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.523971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.559354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1228/2465] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:15 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:15.560909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.585405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:15.601471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a72e1bd69b041fb912353db6c2de079'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.613333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a72e1bd69b041fb912353db6c2de079'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.641314 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a72e1bd69b041fb912353db6c2de079'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.644667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab38285-8af0-49dd-9276-ed08705c40a5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.650249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.651281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.651693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a72e1bd69b041fb912353db6c2de079'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.652071 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.652664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.652864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a72e1bd69b041fb912353db6c2de079'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:15.653798 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.655754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.663213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1238/2466] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:15 2022] DELETE /identity/v3/users/0a72e1bd69b041fb912353db6c2de079 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:15.697737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1229/2467] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:15 2022] POST /identity/v3/users => generated 303 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:15.701954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.706254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:15.800129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff0ff-497d-461c-b303-2ec1b5d33ac1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.812115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.812863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.813619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.813954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.815549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.817552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.833420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1239/2468] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:15.834872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.847289 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382fcf-8842-43bd-8e7f-180a79dfe514 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.852501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.853220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.853581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.853936 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.854829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.856794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1b308-2b8f-4ad7-8640-7248543dbb3f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.875946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1230/2469] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:15.881733 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:15.901474 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc5be6-b14c-474e-8df9-411a1986fac2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:15.906599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:15.907477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.907945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.908834 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] PATH_INFO: `/v3/projects/ef2a0fa7bc1f4744abff7bf6812fa4d7/users/fb3b37e478f84cf3836f91f1ee4ed208/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:15.958824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2a0fa7bc1f4744abff7bf6812fa4d7, user_id=fb3b37e478f84cf3836f91f1ee4ed208, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:15.963671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:15.981726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1240/2470] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:15 2022] PUT /identity/v3/projects/ef2a0fa7bc1f4744abff7bf6812fa4d7/users/fb3b37e478f84cf3836f91f1ee4ed208/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:15.997655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:15.998492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:15.999036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.037840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] MFA Rules not processed for user `fb3b37e478f84cf3836f91f1ee4ed208`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:16.040002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.137755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1231/2471] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:16.143674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.239479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.245094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.261697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c592-a9bb-4b6d-9360-0c33bcc59b18 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.274194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.274846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.275130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.275403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.276125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.278122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.284724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455540fb-a6bf-4fa7-9a60-e92a0e219d5d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.291849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.292613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.292898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.293230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] PATH_INFO: `/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.294526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc7e50586884fd7b68d8fe2a867b3bb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.296593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.318964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:16.350645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1241/2472] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:16.352946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.380931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:16.385919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc7e50586884fd7b68d8fe2a867b3bb', 'request_id': 'req-c70c4a54-7784-449b-9fc9-774fc31b8982'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:16.396347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc7e50586884fd7b68d8fe2a867b3bb', 'request_id': 'req-c70c4a54-7784-449b-9fc9-774fc31b8982'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:16.407950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc7e50586884fd7b68d8fe2a867b3bb', 'request_id': 'req-c70c4a54-7784-449b-9fc9-774fc31b8982'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:16.427042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] The token cache is being invalidate because project acc7e50586884fd7b68d8fe2a867b3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:16.427042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:16.427931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c70c4a54-7784-449b-9fc9-774fc31b8982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:16.429637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1232/2473] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:16 2022] DELETE /identity/v3/projects/acc7e50586884fd7b68d8fe2a867b3bb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:16.434819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bf790-0148-4359-90ea-32b0458e3b4a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.435131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bf790-0148-4359-90ea-32b0458e3b4a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.435449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bf790-0148-4359-90ea-32b0458e3b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.456623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4e5bf790-0148-4359-90ea-32b0458e3b4a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:16.457846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4e5bf790-0148-4359-90ea-32b0458e3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.460531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120e7-25ab-4323-9cb6-b4e7eb48892b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.527241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1233/2474] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:16.533424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.535224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.535619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.536053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.536693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.537171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.537463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.538158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.540761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.559779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:16.561084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.570820 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1234/2475] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:16.572521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1242/2476] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/users => generated 309 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:16.575818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.578306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.580946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.592349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4e025-2763-4857-96d6-55ffb9d11f71 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.593090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd029976-e6a5-405d-8d06-7e82d5a3a759 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.598447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.598984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.599267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.599579 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.600323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.600682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.601724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.602053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.602267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.602767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.604072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:16.604624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.607550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.612529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1243/2477] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:16.617279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.626591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1235/2478] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:16.626912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ca255-0a90-4df6-84f1-0c3cd90abfee admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.632184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.632623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.632879 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.633168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.633446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] PATH_INFO: `/v3/projects/80a96ce8722440368f29e5b2a47e7b89/users/07feb38a87c24b73903cd4424a6917c8/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.643863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390a4d-a5ac-4d7f-b79b-f9ff10f35cf7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.648519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.649135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.649485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.649690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.650361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.650938 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a96ce8722440368f29e5b2a47e7b89, user_id=07feb38a87c24b73903cd4424a6917c8, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.652315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.653950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.671648 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1244/2479] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:16 2022] PUT /identity/v3/projects/80a96ce8722440368f29e5b2a47e7b89/users/07feb38a87c24b73903cd4424a6917c8/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:16.676228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.685503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1236/2480] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:16.689913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.745632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0923260-99a1-4856-9c0a-b75c7fea7c76 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.762312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:16.764099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:16.768417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e708-5fe8-4c7f-9678-581d3c1c3024 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.775000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.775701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.776178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.776639 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.777474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.779533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.801374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1237/2481] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:16.805456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.806916 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.807539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.807821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.808120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.809085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.810691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.815996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8510d0c-2abe-4588-bddb-1382b4330e1b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.820631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.824400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1245/2482] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:16.824569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.824569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.824569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.824569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.824569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.825177 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.834678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1238/2483] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:16.835148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f26ba5-1d3a-4b31-8120-9dd7f1aa60eb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.838546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.839898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.840560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.840894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.841174 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] PATH_INFO: `/v3/projects/80a96ce8722440368f29e5b2a47e7b89/users/07feb38a87c24b73903cd4424a6917c8/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.843200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a96ce8722440368f29e5b2a47e7b89, user_id=07feb38a87c24b73903cd4424a6917c8, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.845929 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467a282-2f3a-4189-9d10-f5ee3622eb1a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.848691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf60b5-5ecf-4c92-87a2-37967abc2454 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.854284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.854842 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.855135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.855411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.856207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.856753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1246/2484] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:16 2022] PUT /identity/v3/projects/80a96ce8722440368f29e5b2a47e7b89/users/07feb38a87c24b73903cd4424a6917c8/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:16.857957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.870135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1239/2485] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:16.870336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123548da-1c20-47f7-9fab-0916ef56a87e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.870698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123548da-1c20-47f7-9fab-0916ef56a87e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.870980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123548da-1c20-47f7-9fab-0916ef56a87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.874636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.884543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ee0b7-4276-4d88-b9f8-3a1f1b0da3fb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:16.899770 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-123548da-1c20-47f7-9fab-0916ef56a87e None None] MFA Rules not processed for user `07feb38a87c24b73903cd4424a6917c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:16.901007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-123548da-1c20-47f7-9fab-0916ef56a87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:16.943498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:16.944164 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.944445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.944716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] PATH_INFO: `/v3/projects/33d5f2cbe7ee436b982a61bb80069522/users/81067bfb66fb4ad7afcc35cc32357f50/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.961176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d5f2cbe7ee436b982a61bb80069522, user_id=81067bfb66fb4ad7afcc35cc32357f50, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:16.964145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:16.975798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1247/2486] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:16.979483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1240/2487] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:16 2022] PUT /identity/v3/projects/33d5f2cbe7ee436b982a61bb80069522/users/81067bfb66fb4ad7afcc35cc32357f50/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:16.989823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:16.990120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:16.990454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:16.990946 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:16.992687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.019588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] MFA Rules not processed for user `81067bfb66fb4ad7afcc35cc32357f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:17.020788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.052124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59daa27-ea94-4e39-b795-76715b9fe0e4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.059916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.060541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.060849 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.061122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.092520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1248/2488] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:17.097899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.125079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:17.159224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.160763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.165020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:17.176018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1241/2489] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:17.182549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eebd0f-4d08-4d1a-8718-f623eec04a6e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.188328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.188979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.189262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.190059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.190819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.192847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.234363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1249/2490] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:17.238728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.250901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd49d48-fc49-4640-871f-6fc16ef09e1e service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.328685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.329310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.329593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.329870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.331066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.333008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.370878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1242/2491] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:17.371120 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.372389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.383366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f76e1-106b-463c-b1d0-e4af7e4c1066 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.384892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171146b7-339f-4dd7-a899-552137b0b8bf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.388400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.389050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.389335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.389613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] PATH_INFO: `/v3/users/fb3b37e478f84cf3836f91f1ee4ed208` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.390809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3b37e478f84cf3836f91f1ee4ed208)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.391959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.393409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.393530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.393895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.394358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.396011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.398501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.416388 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1243/2492] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:17.422117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.441559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e03ef-dbbf-4971-8574-c2d4a35cd126 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.449049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3b37e478f84cf3836f91f1ee4ed208'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:17.449240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.450201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.450652 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.451127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.457527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.457527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.458234 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3b37e478f84cf3836f91f1ee4ed208'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:17.470589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3b37e478f84cf3836f91f1ee4ed208'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:17.475360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1244/2493] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:17.478276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3b37e478f84cf3836f91f1ee4ed208'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:17.479225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b8bdb466-18a2-49a0-93aa-458def45bd97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3b37e478f84cf3836f91f1ee4ed208'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:17.481801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.483993 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.487292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1250/2494] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:17 2022] DELETE /identity/v3/users/fb3b37e478f84cf3836f91f1ee4ed208 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:17.492003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.492297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.492577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.499982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301398-c3c5-48f4-8eb1-d4db1e755365 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.513250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:17.514504 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.573185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.574015 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.574483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.574954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] PATH_INFO: `/v3/projects/7ebf2dd4f1d241a29eeab11ff449fb1e/users/694a7c6f4fd8426592a519b72d1abb87/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.597466 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebf2dd4f1d241a29eeab11ff449fb1e, user_id=694a7c6f4fd8426592a519b72d1abb87, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.597754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1251/2495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:17.601250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab55a0-bf88-4c47-a7db-2b31656ceee5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.612189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.614287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.620488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1245/2496] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:17 2022] PUT /identity/v3/projects/7ebf2dd4f1d241a29eeab11ff449fb1e/users/694a7c6f4fd8426592a519b72d1abb87/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:17.633058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.633861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.634338 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.643693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:17.671700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] MFA Rules not processed for user `694a7c6f4fd8426592a519b72d1abb87`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:17.675122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:17.706543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14e359-6727-4247-b97e-2691217485f3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.718794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:17.749679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1246/2497] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:17.754597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.770036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.770654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.770935 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.771205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.820223 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49f3e0-c4eb-4f66-b045-1fcb53508bf2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.824990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.825549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.825832 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.826107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.826780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.828713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.834516 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.836114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.846982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1252/2498] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:17.859914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1247/2499] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:17.864697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.874990 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95071db0-9e16-4681-b01e-6bc8d6feb332 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.941286 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.942189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.942466 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.942739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.943409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.945537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:17.972554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1253/2500] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:17 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:17.973591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:17.984556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcb02b-8e3f-407a-b526-e105557b4c46 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:17.990738 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:17.991376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:17.991659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:17.991979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:17.992683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:17.994302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.005585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1248/2501] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:18.009806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.021294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc35cd-39c6-4582-b01d-ef017f73c75b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.026071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.026526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.026865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.027032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] PATH_INFO: `/v3/projects/c29fff13464849e482cf73087ecff0e6/users/5136db972ea248d3b5ec234a73135508/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.037560 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:18.046184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29fff13464849e482cf73087ecff0e6, user_id=5136db972ea248d3b5ec234a73135508, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.049292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.066157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1254/2502] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:18 2022] PUT /identity/v3/projects/c29fff13464849e482cf73087ecff0e6/users/5136db972ea248d3b5ec234a73135508/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:18.072426 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.085455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896119-07ec-4e3e-8a41-98e679b597aa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.151425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.152710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.153205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.153702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.154961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.159362 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.182882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1249/2503] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:18.185625 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.186777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.190460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:18.198406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ce6d-c16e-4efb-909d-872f0b423235 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.203821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.204665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.205467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.205743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] PATH_INFO: `/v3/projects/c29fff13464849e482cf73087ecff0e6/users/5136db972ea248d3b5ec234a73135508/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.207806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29fff13464849e482cf73087ecff0e6, user_id=5136db972ea248d3b5ec234a73135508, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.211130 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da093b-879b-4d6a-95c1-3220ba1be1bb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.215283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e42ad-94db-438e-9796-a09a6ea87fbf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.223969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1255/2504] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:18 2022] PUT /identity/v3/projects/c29fff13464849e482cf73087ecff0e6/users/5136db972ea248d3b5ec234a73135508/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:18.234203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.234468 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.234753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.269611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:18.271351 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] MFA Rules not processed for user `5136db972ea248d3b5ec234a73135508`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:18.272922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:18.284559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.285340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.285544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.285821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] PATH_INFO: `/v3/projects/ef2a0fa7bc1f4744abff7bf6812fa4d7` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.287013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2a0fa7bc1f4744abff7bf6812fa4d7)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.289184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.342416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2a0fa7bc1f4744abff7bf6812fa4d7', 'request_id': 'req-12a877bc-486f-4b0c-aacc-8af18761b99c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.349833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1256/2505] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:18.352705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2a0fa7bc1f4744abff7bf6812fa4d7', 'request_id': 'req-12a877bc-486f-4b0c-aacc-8af18761b99c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.362190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.362793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2a0fa7bc1f4744abff7bf6812fa4d7', 'request_id': 'req-12a877bc-486f-4b0c-aacc-8af18761b99c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.363688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:18.374399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] The token cache is being invalidate because project ef2a0fa7bc1f4744abff7bf6812fa4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:18.374716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.375309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.376944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1250/2506] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:18 2022] DELETE /identity/v3/projects/ef2a0fa7bc1f4744abff7bf6812fa4d7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:18.381477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.383029 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:18.432224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e43e-cdac-49f1-ace9-7f821b50fbbc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.453017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a877bc-486f-4b0c-aacc-8af18761b99c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:18.458369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.459045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.459279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.459554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] PATH_INFO: `/v3/users/07feb38a87c24b73903cd4424a6917c8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.460795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07feb38a87c24b73903cd4424a6917c8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.462717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.484200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:18.484835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:18.485117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:18.485392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:18.488317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:18.522178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07feb38a87c24b73903cd4424a6917c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.522411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:18.523209 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07feb38a87c24b73903cd4424a6917c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.532103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07feb38a87c24b73903cd4424a6917c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.540349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07feb38a87c24b73903cd4424a6917c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.553816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07feb38a87c24b73903cd4424a6917c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:18.555474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:18.559761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:18.561800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1251/2507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:18 2022] DELETE /identity/v3/users/07feb38a87c24b73903cd4424a6917c8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:18.562679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9311ec6-1f4d-4282-9f26-7517267a0882 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:18.581721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1257/2508] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:18.935054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:18.936822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.114858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d4b1c-f192-45a5-bce9-3e05660685a2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.114858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.114858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.114858 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.116815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] PATH_INFO: `/v3/projects/80a96ce8722440368f29e5b2a47e7b89` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.116815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a96ce8722440368f29e5b2a47e7b89)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.116815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.143534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a96ce8722440368f29e5b2a47e7b89', 'request_id': 'req-bb3adbf4-f4fb-40c5-8eca-474716eba34a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:19.151710 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a96ce8722440368f29e5b2a47e7b89', 'request_id': 'req-bb3adbf4-f4fb-40c5-8eca-474716eba34a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:19.161807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a96ce8722440368f29e5b2a47e7b89', 'request_id': 'req-bb3adbf4-f4fb-40c5-8eca-474716eba34a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:19.174031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] The token cache is being invalidate because project 80a96ce8722440368f29e5b2a47e7b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:19.174323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:19.175059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-bb3adbf4-f4fb-40c5-8eca-474716eba34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:19.176687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1252/2509] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:18 2022] DELETE /identity/v3/projects/80a96ce8722440368f29e5b2a47e7b89 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:19.183285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.184423 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.184545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.208773 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:19.210471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.283539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1258/2510] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:19.291370 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.291659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.292032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.316556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:19.318326 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.332649 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1253/2511] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:19.339765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.349864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4afe70-7533-4b67-82c4-ed2603e35fe7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.354741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.355364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.355640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.355944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.356651 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:19.356955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.358599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.371865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1259/2512] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:19.376221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.391375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87da7ec-9e82-4c38-88f0-95f07b0262d1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.396986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.397615 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.397944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.398219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.398995 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.401028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.401325 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.403854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.436758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1254/2513] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:19.441569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.488025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a4bc-9d61-48bb-94c7-079e770d1f3c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.519554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c0100-b611-4acd-b9ee-973eca0b0c15 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.524509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.525186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.525474 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.525773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.526486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.528461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.537032 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:19.557401 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.558084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1255/2514] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:19.558520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.559368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.559900 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.562314 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.563024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.572794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9adb48-48e0-4823-aa26-8bf13c690512 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.577982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.578606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.578888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.579166 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.580442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.582062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.592524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1256/2515] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:19.599860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:19.606095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.621228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651f978-e2a7-4cc8-874a-046d609a2c3b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.645022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.645276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.645684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.645981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.646259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.646687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.647070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.648759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e53c0-c01e-40ed-9474-0a0c3049236b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.657151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1260/2516] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:19.662476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1257/2517] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:19.668438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.681591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0684e0-3214-4f89-a52f-f2b0ccc6a553 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.686974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.687669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.688008 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.688287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] PATH_INFO: `/v3/projects/08d3beeba6ce451eaf57b6a1a58cd330/users/3324a3d0ed65406ba427f219b14014a3/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.705680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d3beeba6ce451eaf57b6a1a58cd330, user_id=3324a3d0ed65406ba427f219b14014a3, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.708442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:19.727197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1261/2518] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:19 2022] PUT /identity/v3/projects/08d3beeba6ce451eaf57b6a1a58cd330/users/3324a3d0ed65406ba427f219b14014a3/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:19.745749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.746025 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.746312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.792268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] MFA Rules not processed for user `3324a3d0ed65406ba427f219b14014a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:19.793496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:19.877449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1258/2519] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:19.882677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:19.969056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb866c-8587-4dc5-a581-38d9f979adb6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:19.974477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:19.975113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:19.975403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:19.975673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:19.976403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:19.978349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.016713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1262/2520] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:20.021396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.041865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225544a2-21a5-4ecb-8192-74df9e491c3d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.129638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.130488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.130877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.131250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.132227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.135084 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.173387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1259/2521] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:20 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:20.178635 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.194088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d47ad-5fc7-405e-b474-e69931cc5a2f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.199478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.200256 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.200639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.201029 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.201805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.203663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.217135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1263/2522] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:20.223578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.236309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac46c15-0c88-41cd-b467-276206d72088 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.240980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.241645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.241929 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.242199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] PATH_INFO: `/v3/projects/888af6270df6482fa138bfdcc2006a96/users/4b7fb08ee2bf415d8a39183c446679c2/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.258282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888af6270df6482fa138bfdcc2006a96, user_id=4b7fb08ee2bf415d8a39183c446679c2, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.261085 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.279664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1260/2523] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:20 2022] PUT /identity/v3/projects/888af6270df6482fa138bfdcc2006a96/users/4b7fb08ee2bf415d8a39183c446679c2/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:20.283986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.297819 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759f5db-bfec-4ebe-a013-8119417eaf8e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.368042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.368680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.368969 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.369247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.369955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.371575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f260eac-eb7c-4769-8575-184036929486 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.388158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1264/2524] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:20.392230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.403558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6dbaf-929f-479a-a249-2c08c2150bc7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.408336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.411351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.411351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.411351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] PATH_INFO: `/v3/projects/888af6270df6482fa138bfdcc2006a96/users/4b7fb08ee2bf415d8a39183c446679c2/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.411740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=888af6270df6482fa138bfdcc2006a96, user_id=4b7fb08ee2bf415d8a39183c446679c2, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.414213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.425915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1261/2525] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:20 2022] PUT /identity/v3/projects/888af6270df6482fa138bfdcc2006a96/users/4b7fb08ee2bf415d8a39183c446679c2/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:20.442829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.443116 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.443426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.464764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:20.477393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] MFA Rules not processed for user `4b7fb08ee2bf415d8a39183c446679c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:20.478631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:20.547628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1265/2526] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:20.560413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.571214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:20.625590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287adf76-300f-4b69-a94d-8091cad8f05d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.631082 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.631948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.632349 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.632735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.644837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.646357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747b01-9e7a-416f-b51b-444c61c27140 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.656313 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1262/2527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:20.687618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.751413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fddc34-88f7-45ce-b00a-1958bd35b314 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:20.756429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:20.757076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.757365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.757645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] PATH_INFO: `/v3/users/3324a3d0ed65406ba427f219b14014a3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.758846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3324a3d0ed65406ba427f219b14014a3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:20.760849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:20.823168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3324a3d0ed65406ba427f219b14014a3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:20.837419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3324a3d0ed65406ba427f219b14014a3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:20.856799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3324a3d0ed65406ba427f219b14014a3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:20.867495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3324a3d0ed65406ba427f219b14014a3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:20.868426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3324a3d0ed65406ba427f219b14014a3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:20.874933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1266/2528] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:20 2022] DELETE /identity/v3/users/3324a3d0ed65406ba427f219b14014a3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:20.879613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:20.880002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:20.880328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:20.900188 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:20.901430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:20.968225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1263/2529] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:20.978693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:20.980200 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:20.995531 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:21.049182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec4eb9-5437-4174-96d8-d8ef87bf7f8a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:21.053191 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:21.053821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:21.054160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:21.054440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:21.065158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:21.066612 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:21.082827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1267/2530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:21.436153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:21.437898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:21.452220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d46246-7d94-43ad-aa98-94b9c914b0c7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:21.458181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:21.459088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:21.459489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:21.459762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] PATH_INFO: `/v3/projects/08d3beeba6ce451eaf57b6a1a58cd330` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:21.461089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d3beeba6ce451eaf57b6a1a58cd330)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:21.463020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:21.521045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d3beeba6ce451eaf57b6a1a58cd330', 'request_id': 'req-a48372dc-a69c-40f1-9545-01dc41b073e9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.529501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d3beeba6ce451eaf57b6a1a58cd330', 'request_id': 'req-a48372dc-a69c-40f1-9545-01dc41b073e9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.540109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d3beeba6ce451eaf57b6a1a58cd330', 'request_id': 'req-a48372dc-a69c-40f1-9545-01dc41b073e9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.556336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] The token cache is being invalidate because project 08d3beeba6ce451eaf57b6a1a58cd330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:21.556625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.557284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.559081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1264/2531] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:21 2022] DELETE /identity/v3/projects/08d3beeba6ce451eaf57b6a1a58cd330 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:21.563601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:21.574845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94e46-c842-4d73-afa4-8339c80b35af service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:21.678705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:21.679503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:21.679784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:21.680088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] PATH_INFO: `/v3/users/4b7fb08ee2bf415d8a39183c446679c2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:21.681344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7fb08ee2bf415d8a39183c446679c2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:21.683285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:21.742508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7fb08ee2bf415d8a39183c446679c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.753022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7fb08ee2bf415d8a39183c446679c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.772636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7fb08ee2bf415d8a39183c446679c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.783795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7fb08ee2bf415d8a39183c446679c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.784845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b3859728-03e0-46dd-b95d-4d9c405af4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7fb08ee2bf415d8a39183c446679c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:21.793846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1268/2532] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:21 2022] DELETE /identity/v3/users/4b7fb08ee2bf415d8a39183c446679c2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:22.093596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.096107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:22.112476 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48372dc-a69c-40f1-9545-01dc41b073e9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.147526 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:22.174421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.175064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.175350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.175630 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] PATH_INFO: `/v3/projects/888af6270df6482fa138bfdcc2006a96` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.176843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=888af6270df6482fa138bfdcc2006a96)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.178751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.226228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888af6270df6482fa138bfdcc2006a96', 'request_id': 'req-db3fef01-17b7-4087-aee9-c4f65b47bbf9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:22.232436 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888af6270df6482fa138bfdcc2006a96', 'request_id': 'req-db3fef01-17b7-4087-aee9-c4f65b47bbf9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:22.240754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888af6270df6482fa138bfdcc2006a96', 'request_id': 'req-db3fef01-17b7-4087-aee9-c4f65b47bbf9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:22.252727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] The token cache is being invalidate because project 888af6270df6482fa138bfdcc2006a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:22.253020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:22.253677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db3fef01-17b7-4087-aee9-c4f65b47bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:22.255254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1265/2533] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:22 2022] DELETE /identity/v3/projects/888af6270df6482fa138bfdcc2006a96 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:22.260132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.260644 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.260868 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.281193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:22.282396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:22.314914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:22.352396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1269/2534] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:22.360086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.360382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.360684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.382242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:22.383462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:22.391860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1266/2535] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:22.396691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.406903 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7321b-8ca5-4044-95e8-7ec778493664 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.412739 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.413395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.413676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.413955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.414658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:22.414964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.416632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.430288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1270/2536] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:22.435372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.456566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f383e16-308e-4d97-a6ac-fc81825fd7ee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.464023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.465342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.465590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.465978 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.467009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.469976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.504028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1267/2537] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:22.509106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.523026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4767-6b1c-4226-b719-3dff9fdd2abc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.617181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.618449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.618449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.619062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.619941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.621985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.649666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1271/2538] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:22.653931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.667698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d945cbc-60cf-4d8e-8dc0-ab3ebbf69f80 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.674363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.675750 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.676268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.676809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.677578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.679514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.695622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1268/2539] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:22.701420 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.716172 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545170-55fe-4c1e-8bb6-d23b003d0132 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.723119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.724051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.724479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.724871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.726502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.728937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdf931-31fa-447d-a00b-a78b6801b202 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.747896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1272/2540] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:22.752955 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:22.764839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b128-57ec-4ce1-b4be-123938d97b4e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:22.771406 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:22.772124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.772410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.772692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] PATH_INFO: `/v3/projects/8a3899ff5d2b4b5486db637cccf4b6c8/users/1fcd07cfaf574e5193aa85d389911be2/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.794912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3899ff5d2b4b5486db637cccf4b6c8, user_id=1fcd07cfaf574e5193aa85d389911be2, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:22.799011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:22.815768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1269/2541] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:22 2022] PUT /identity/v3/projects/8a3899ff5d2b4b5486db637cccf4b6c8/users/1fcd07cfaf574e5193aa85d389911be2/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:22.831429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:22.831713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:22.832041 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:22.864881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] MFA Rules not processed for user `1fcd07cfaf574e5193aa85d389911be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:22.866279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:22.954641 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1273/2542] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:22.960456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.029897 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0201a45-b341-4a4a-b278-ddad0a60d2d0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.034788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.035416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.035697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.036006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.036677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.038611 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.044489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:23.077641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1270/2543] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:22 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:23.082397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.093911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04217b-0327-4ece-a565-6a61047cfea0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.124933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:23.175782 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.176437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.176715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.176986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.177850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.179775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.203971 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1274/2544] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:23 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:23.208844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.219464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e241dc-1726-4a14-8c12-2fe6cb5ab0b8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.224256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.224883 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.225264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.225537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.226224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.227868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.238044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1271/2545] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:23.244000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.255089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97302ab-8d18-49d2-95d0-8d750819978b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.259919 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.260592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.260880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.261226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] PATH_INFO: `/v3/projects/d7539230ea3f420291b123f6b42d40b0/users/209c7ae747a446c6a36c36c3618dd84c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.278569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7539230ea3f420291b123f6b42d40b0, user_id=209c7ae747a446c6a36c36c3618dd84c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.281400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.294902 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1275/2546] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:23 2022] PUT /identity/v3/projects/d7539230ea3f420291b123f6b42d40b0/users/209c7ae747a446c6a36c36c3618dd84c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:23.299721 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.311265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8ed7a-3695-4da9-a461-ad9c33e01995 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.370548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.371170 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.371454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.371732 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.372463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.374066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a1d0f-017c-4b76-bc7a-e3e0a7561a63 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.391729 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1272/2547] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:23.396897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.410546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d34be-b8b5-4f6e-adaa-d4974672d747 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.415403 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.416117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.416402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.416680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] PATH_INFO: `/v3/projects/d7539230ea3f420291b123f6b42d40b0/users/209c7ae747a446c6a36c36c3618dd84c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.418699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7539230ea3f420291b123f6b42d40b0, user_id=209c7ae747a446c6a36c36c3618dd84c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.421415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.434337 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1276/2548] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:23 2022] PUT /identity/v3/projects/d7539230ea3f420291b123f6b42d40b0/users/209c7ae747a446c6a36c36c3618dd84c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:23.448099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.448375 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.448659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.479046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] MFA Rules not processed for user `209c7ae747a446c6a36c36c3618dd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:23.480451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:23.551057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1273/2549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:23.562486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.564757 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:23.627941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715bdc-656b-4910-9b69-b27bbfd639e5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.631913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.632545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.632833 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.633113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.643159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:23.644640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:23.654650 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1277/2550] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:23.925801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:23.927383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:23.941039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cbd4a-7f72-4d0a-87b2-a9ad36dd7314 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:23.945135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:23.945942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:23.946320 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:23.946692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:23.948838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:24.022438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:24.025723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:24.028197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d27b88-51bb-4dce-adad-d78db2c46493 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:24.040119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1274/2551] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:24.261126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:24.353652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705fb3e-3c86-43fe-97ca-fa123de6f3a9 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:24.358987 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:24.359665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:24.359980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:24.360260 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] PATH_INFO: `/v3/users/1fcd07cfaf574e5193aa85d389911be2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:24.361603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fcd07cfaf574e5193aa85d389911be2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:24.363572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:24.419363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:24.456460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fcd07cfaf574e5193aa85d389911be2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:24.466764 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fcd07cfaf574e5193aa85d389911be2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:24.490852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fcd07cfaf574e5193aa85d389911be2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:24.503158 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fcd07cfaf574e5193aa85d389911be2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:24.503968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fcd07cfaf574e5193aa85d389911be2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:24.511244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1278/2552] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:24 2022] DELETE /identity/v3/users/1fcd07cfaf574e5193aa85d389911be2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:24.517053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:24.518137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:24.518137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:24.552479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:24.555429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:24.667578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1275/2553] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:24.676488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:24.678537 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:24.765321 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97f714-e679-4324-a225-a2d912fa5e1c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:24.769772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:24.770491 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:24.770869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:24.771219 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:24.783693 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:24.785421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:24.799787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1279/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:25.170096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:25.171711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:25.186407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41770f6-173b-46dd-9266-094ebc0b0399 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:25.192664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:25.193475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:25.193968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:25.194695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] PATH_INFO: `/v3/projects/8a3899ff5d2b4b5486db637cccf4b6c8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:25.196018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3899ff5d2b4b5486db637cccf4b6c8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:25.198280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:25.262694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3899ff5d2b4b5486db637cccf4b6c8', 'request_id': 'req-18178886-c623-4214-9dfe-d30d83dfb5b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.271517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3899ff5d2b4b5486db637cccf4b6c8', 'request_id': 'req-18178886-c623-4214-9dfe-d30d83dfb5b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.282665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3899ff5d2b4b5486db637cccf4b6c8', 'request_id': 'req-18178886-c623-4214-9dfe-d30d83dfb5b5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.295935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] The token cache is being invalidate because project 8a3899ff5d2b4b5486db637cccf4b6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:25.296360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.297310 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.299815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1276/2555] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:25 2022] DELETE /identity/v3/projects/8a3899ff5d2b4b5486db637cccf4b6c8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:25.303957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:25.316136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b70c9-d59a-4bb2-8514-a0e759a56882 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:25.385901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:25.386552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:25.386835 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:25.387114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] PATH_INFO: `/v3/users/209c7ae747a446c6a36c36c3618dd84c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:25.388323 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=209c7ae747a446c6a36c36c3618dd84c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:25.390255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:25.445804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209c7ae747a446c6a36c36c3618dd84c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.450006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:25.457123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209c7ae747a446c6a36c36c3618dd84c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.471639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209c7ae747a446c6a36c36c3618dd84c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.478703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209c7ae747a446c6a36c36c3618dd84c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.479411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-93239be7-08bd-490c-8e27-f90d50f62f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209c7ae747a446c6a36c36c3618dd84c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.485387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1280/2556] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:25 2022] DELETE /identity/v3/users/209c7ae747a446c6a36c36c3618dd84c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:25.776581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:25.778111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:25.792533 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18178886-c623-4214-9dfe-d30d83dfb5b5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:25.862152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:25.862819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:25.863103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:25.863396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] PATH_INFO: `/v3/projects/d7539230ea3f420291b123f6b42d40b0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:25.864792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7539230ea3f420291b123f6b42d40b0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:25.866755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:25.890572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:25.916178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7539230ea3f420291b123f6b42d40b0', 'request_id': 'req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.922979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7539230ea3f420291b123f6b42d40b0', 'request_id': 'req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.932427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7539230ea3f420291b123f6b42d40b0', 'request_id': 'req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.945298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] The token cache is being invalidate because project d7539230ea3f420291b123f6b42d40b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:25.945603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.946311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a2ca2af0-81bf-49c2-be6a-965bea43bbf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:25.947964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1277/2557] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:25 2022] DELETE /identity/v3/projects/d7539230ea3f420291b123f6b42d40b0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:25.953540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:25.954059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:25.954265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:25.976419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:25.977532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.040914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1281/2558] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:26.052692 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.053009 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.053299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.080360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:26.082054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.091959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1278/2559] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:26.099085 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.109361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf09c-dbc6-4b3d-ba4a-15bb3c2aa2c6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.114861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.115483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.115765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.116080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.116898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:26.117208 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.118837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.133163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1282/2560] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:26.138288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.149566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1017af-7892-4651-b04e-50cf1c88ff06 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.154591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.155444 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.155625 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.156031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.156773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.159511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.199495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1279/2561] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:26 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:26.203487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.215383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78ccf-7c7b-4542-ab86-e9e7eb58be9e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.280154 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.280821 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.281117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.281429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.282179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.284815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.309487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1283/2562] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:26 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:26.314087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.325005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472aa5-a8c1-4a0f-a9a5-3d2980e9639a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.330699 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.331465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.331756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.332065 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.332764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.334372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.349641 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1280/2563] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:26.350983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.361387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92762f5-e2b9-4bb6-bc65-572c46bd8c80 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.366813 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.367442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.367731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.368057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.368759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.370771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41a93-be22-4687-89f4-f3209a118732 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.382758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1284/2564] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:26.388677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.399610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aeac0-a076-480d-9a12-41566fd9723b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.404685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.405361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.405695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.405976 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] PATH_INFO: `/v3/projects/d4fad8f265cd458b8557cf0d21adfa1f/users/f7c451926101491caa1df6e7cabe118a/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.423064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4fad8f265cd458b8557cf0d21adfa1f, user_id=f7c451926101491caa1df6e7cabe118a, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.425888 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.441752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1281/2565] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:26 2022] PUT /identity/v3/projects/d4fad8f265cd458b8557cf0d21adfa1f/users/f7c451926101491caa1df6e7cabe118a/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:26.449229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.449750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.450115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.470701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:26.472161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.504555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:26.513818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.515627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.542815 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1285/2566] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:26.553093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.554454 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.585072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef6f67-1fdd-4f46-bf5c-044c349fd785 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.589780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.590422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.590704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.590982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] PATH_INFO: `/v3/users/81067bfb66fb4ad7afcc35cc32357f50` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.592662 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81067bfb66fb4ad7afcc35cc32357f50)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.594574 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.614035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52aaea-d3cc-4d25-8f20-f21a96265e05 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.618528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.619189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.619445 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.619718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.632744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.634205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305d392-3857-44ba-9781-0a1b51e72f8f service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.643865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81067bfb66fb4ad7afcc35cc32357f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:26.644270 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1286/2567] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:26.644823 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81067bfb66fb4ad7afcc35cc32357f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:26.646612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:26.654168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81067bfb66fb4ad7afcc35cc32357f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:26.664910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81067bfb66fb4ad7afcc35cc32357f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:26.678964 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81067bfb66fb4ad7afcc35cc32357f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:26.685219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1282/2568] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:26 2022] DELETE /identity/v3/users/81067bfb66fb4ad7afcc35cc32357f50 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:26.690776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.691069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.691346 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.711694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:26.712946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.799580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1287/2569] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:26.813595 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:26.815932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:26.885654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1f02c-9f0c-4689-b75a-e49346215bd4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:26.889616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:26.890261 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:26.890541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:26.890819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:26.912565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:26.914093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf2aea-526c-4924-82da-7ff6399c278c service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:26.927608 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1283/2570] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:27.349304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.350883 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:27.366182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d420865-6599-4c2e-811b-6d09603155f1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.383210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.384126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.384528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.385024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] PATH_INFO: `/v3/projects/33d5f2cbe7ee436b982a61bb80069522` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.386583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d5f2cbe7ee436b982a61bb80069522)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.389628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.450262 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d5f2cbe7ee436b982a61bb80069522', 'request_id': 'req-5b4aa49a-6e54-42d7-99a6-2afec7412741'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.461019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d5f2cbe7ee436b982a61bb80069522', 'request_id': 'req-5b4aa49a-6e54-42d7-99a6-2afec7412741'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.468457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.468773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.469101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.474571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d5f2cbe7ee436b982a61bb80069522', 'request_id': 'req-5b4aa49a-6e54-42d7-99a6-2afec7412741'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.487298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:27.495545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] The token cache is being invalidate because project 33d5f2cbe7ee436b982a61bb80069522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:27.497396 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.499524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.506807 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1288/2571] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:27 2022] DELETE /identity/v3/projects/33d5f2cbe7ee436b982a61bb80069522 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:27.508698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.508844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] MFA Rules not processed for user `f7c451926101491caa1df6e7cabe118a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:27.510949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:27.511252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:27.589505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa49a-6e54-42d7-99a6-2afec7412741 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.595723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.599982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1284/2572] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:27.600104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.600104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.600104 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] PATH_INFO: `/v3/users/694a7c6f4fd8426592a519b72d1abb87` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.600456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.600765 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=694a7c6f4fd8426592a519b72d1abb87)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.602921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.604803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:27.619751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff7297-0dce-4b87-af10-a380c7678546 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.626489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.627250 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.627636 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.628055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.628756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.631238 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.663417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694a7c6f4fd8426592a519b72d1abb87'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.663670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1285/2573] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:27 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:27.667501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.680784 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694a7c6f4fd8426592a519b72d1abb87'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.701096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694a7c6f4fd8426592a519b72d1abb87'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.709430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:27.714599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694a7c6f4fd8426592a519b72d1abb87'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.715353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694a7c6f4fd8426592a519b72d1abb87'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:27.722214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1289/2574] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:27 2022] DELETE /identity/v3/users/694a7c6f4fd8426592a519b72d1abb87 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:27.746030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab326d75-50a6-46c0-b9cb-7f2e5104f8be admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.816102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.817003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.817216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.817505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.818187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.820242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.842632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1286/2575] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:27 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:27.846587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.848271 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:27.864761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa9702-5efe-4c18-8abe-1383b888be21 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.869884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.870670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.870958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.871239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.871994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.873653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.887624 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1290/2576] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:27.891441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.902855 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71529be-f9a4-43fa-b786-b58a436fd254 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:27.908213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:27.909441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:27.909441 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:27.909678 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] PATH_INFO: `/v3/projects/c643ea4cc37b45eea4346404772e46a7/users/9e5240923c8746b082b2e22af898541c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:27.934446 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c643ea4cc37b45eea4346404772e46a7, user_id=9e5240923c8746b082b2e22af898541c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:27.937227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:27.957317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1287/2577] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:27 2022] PUT /identity/v3/projects/c643ea4cc37b45eea4346404772e46a7/users/9e5240923c8746b082b2e22af898541c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:27.961436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:27.972956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed998-b49e-4d4e-b836-a58c36373d5f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:28.036274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:28.037012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:28.037299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:28.037605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:28.038484 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:28.040204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:28.050425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:28.052117 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:28.053705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1291/2578] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:28.058926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:28.068275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:28.069207 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e80a5-3406-485b-9fe4-707b42f265ed admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:28.072985 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cc1cf-a4ca-4fb5-9448-c6b3a49aec43 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:28.074998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:28.075756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:28.076093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:28.076377 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] PATH_INFO: `/v3/projects/7ebf2dd4f1d241a29eeab11ff449fb1e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:28.077813 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ebf2dd4f1d241a29eeab11ff449fb1e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:28.080688 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:28.081471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:28.083783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:28.084125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:28.084405 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] PATH_INFO: `/v3/projects/c643ea4cc37b45eea4346404772e46a7/users/9e5240923c8746b082b2e22af898541c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:28.087892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c643ea4cc37b45eea4346404772e46a7, user_id=9e5240923c8746b082b2e22af898541c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:28.092218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:28.108744 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1292/2579] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:28 2022] PUT /identity/v3/projects/c643ea4cc37b45eea4346404772e46a7/users/9e5240923c8746b082b2e22af898541c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:28.154100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ebf2dd4f1d241a29eeab11ff449fb1e', 'request_id': 'req-43645362-d4e9-43f5-a939-c8e6ebab6164'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.162378 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ebf2dd4f1d241a29eeab11ff449fb1e', 'request_id': 'req-43645362-d4e9-43f5-a939-c8e6ebab6164'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.172718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ebf2dd4f1d241a29eeab11ff449fb1e', 'request_id': 'req-43645362-d4e9-43f5-a939-c8e6ebab6164'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.190232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] The token cache is being invalidate because project 7ebf2dd4f1d241a29eeab11ff449fb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:28.190719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.192094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.193623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1288/2580] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:28 2022] DELETE /identity/v3/projects/7ebf2dd4f1d241a29eeab11ff449fb1e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:28.198279 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:28.200067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:28.273440 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63448e4-7f57-4b27-80c9-835cd6f9a7c9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:28.339478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:28.340303 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:28.340580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:28.340855 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] PATH_INFO: `/v3/users/5136db972ea248d3b5ec234a73135508` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:28.342044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5136db972ea248d3b5ec234a73135508)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:28.344095 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:28.393848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5136db972ea248d3b5ec234a73135508'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.401701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5136db972ea248d3b5ec234a73135508'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.415061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5136db972ea248d3b5ec234a73135508'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.422068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5136db972ea248d3b5ec234a73135508'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.422800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-30ff8460-acc1-4075-a570-9967c5d12b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5136db972ea248d3b5ec234a73135508'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:28.430716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1293/2581] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:28 2022] DELETE /identity/v3/users/5136db972ea248d3b5ec234a73135508 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:28.865960 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:28.882312 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43645362-d4e9-43f5-a939-c8e6ebab6164 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:28.912093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:28.912447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:28.912794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:28.928633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:28.959776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:28.978721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] MFA Rules not processed for user `9e5240923c8746b082b2e22af898541c`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:28.980858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.029151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.030033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.030347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.030601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] PATH_INFO: `/v3/projects/c29fff13464849e482cf73087ecff0e6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.031907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29fff13464849e482cf73087ecff0e6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.033874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.067266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1294/2582] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:29.080117 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.081731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.102449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29fff13464849e482cf73087ecff0e6', 'request_id': 'req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:29.111672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29fff13464849e482cf73087ecff0e6', 'request_id': 'req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:29.121279 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29fff13464849e482cf73087ecff0e6', 'request_id': 'req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:29.148284 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] The token cache is being invalidate because project c29fff13464849e482cf73087ecff0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:29.148702 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:29.149410 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-40ee0f6c-ab48-453c-ba2c-7002d24c30d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:29.151276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ced9f-589b-40f1-b2fc-87dbb7915939 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.151401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1289/2583] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:28 2022] DELETE /identity/v3/projects/c29fff13464849e482cf73087ecff0e6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:29.157323 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59bc4-fee3-4188-99e5-716df118fc27 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.157822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59bc4-fee3-4188-99e5-716df118fc27 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.158107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59bc4-fee3-4188-99e5-716df118fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.219018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8ae59bc4-fee3-4188-99e5-716df118fc27 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:29.220234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8ae59bc4-fee3-4188-99e5-716df118fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.223527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.224190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.224479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.224754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.226820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.253401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:29.284213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1290/2584] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:29.297062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:29.298666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.298959 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.299251 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.299426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.301656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.314580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1295/2585] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:29.321105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:29.322478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.330755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1291/2586] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:29.335112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.345988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58b8c9-7651-4d01-9ed2-cff0ddb842df service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.351013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.351852 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.352142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.352778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.353482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:29.353790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.355501 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.371659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1296/2587] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:29.372829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.382922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d77b4-232d-4544-9ed2-d10d52fc0923 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.387797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.388332 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.388613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.388886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.389553 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.391456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.419953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1292/2588] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:29.428788 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.443529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567138c-dbd8-4ab1-ab42-dfed533b8211 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.512464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.513091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.513380 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.513659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.514342 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.516297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.538404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1297/2589] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:29.542707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.553707 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af79bd-544e-4d00-89d3-b8220e04fc62 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.559357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.560011 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.560316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.560596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.561296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.562997 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.574785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1293/2590] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:29.579069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.590026 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1ba2f-03d7-405f-982a-ee97ed3f208f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.594755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.595373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.599724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.600109 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.601111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.602851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6d2b8-6db8-4e6c-a3a9-54f881835602 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.616164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1298/2591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:29.624007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.635922 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc71f-079c-4f59-a1d1-653cb63f5a4f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.640847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.641622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.641998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.642499 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] PATH_INFO: `/v3/projects/cc4d2fba0a7b4e74bb368e0f75548f84/users/66216ab8e87a4723a9ac450d61d61b4b/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.661786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4d2fba0a7b4e74bb368e0f75548f84, user_id=66216ab8e87a4723a9ac450d61d61b4b, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.664666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.685563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1294/2592] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:29 2022] PUT /identity/v3/projects/cc4d2fba0a7b4e74bb368e0f75548f84/users/66216ab8e87a4723a9ac450d61d61b4b/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:29.754599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.754794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.755061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.782153 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] MFA Rules not processed for user `66216ab8e87a4723a9ac450d61d61b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:29.783889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:29.850937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1299/2593] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:29.856077 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.922866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fe758-ee89-4a75-af39-b3c54f0817d5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:29.928580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:29.929225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:29.929537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:29.929825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:29.930800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:29.933631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:29.963111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1295/2594] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:29.968241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:29.978950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1133f7-a687-4490-aa3d-64d929607208 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.032185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:30.043489 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.044168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.044444 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.044721 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.045404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.047327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.072404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1300/2595] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:29 2022] POST /identity/v3/users => generated 307 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:30.076728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.088376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de191-e1cb-45e1-81c1-ad4c64d04895 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.092957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.093576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.093852 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.094124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.094873 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.097047 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.116053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1296/2596] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:30.120456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.131093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1abb2-5d55-4025-acf4-9f14552509ac admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.135943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.136619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.136898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.137204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] PATH_INFO: `/v3/projects/7b404265822a420a9356a10f9469bd33/users/2559cd3819314157a6966f570f0cecca/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.152738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b404265822a420a9356a10f9469bd33, user_id=2559cd3819314157a6966f570f0cecca, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.155496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.170557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1301/2597] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:30 2022] PUT /identity/v3/projects/7b404265822a420a9356a10f9469bd33/users/2559cd3819314157a6966f570f0cecca/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:30.174794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.185201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e18ed-b7b0-4a7d-9739-bc02fd53810f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.232674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:30.246517 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.247141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.247425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.247697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.248434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.250051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fefa9-c752-4857-b5b7-b6d485f0ee12 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.261371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1297/2598] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:30.266185 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.277418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397b26-0435-4253-bfe2-3c1899eb92aa admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.282927 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.283538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.283820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.284132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] PATH_INFO: `/v3/projects/7b404265822a420a9356a10f9469bd33/users/2559cd3819314157a6966f570f0cecca/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.286163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b404265822a420a9356a10f9469bd33, user_id=2559cd3819314157a6966f570f0cecca, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.288893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.299820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1302/2599] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:30 2022] PUT /identity/v3/projects/7b404265822a420a9356a10f9469bd33/users/2559cd3819314157a6966f570f0cecca/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:30.310287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.310579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.310856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.339448 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] MFA Rules not processed for user `2559cd3819314157a6966f570f0cecca`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:30.340711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:30.412461 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1298/2600] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:30.423720 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.425481 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:30.490689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9f72e-10b3-414f-bd11-83f2cc3ba3aa admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.494376 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.494995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.495276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.495548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.505255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.506701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.516412 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1303/2601] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:30.593825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.603728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1dc52-f68d-48db-a7c1-69333a6a2329 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.607434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.608094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.608383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.608665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.667056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.668520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.677965 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1299/2602] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:30.741447 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:30.742995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:30.755621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7754380-11d4-4be7-92f6-c96b8df5721a service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:30.759618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:30.760269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:30.760547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:30.761067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:30.762839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:30.840563 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:30.842066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770de36-499e-4a61-bd1c-9ca7d8328642 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:30.855533 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1304/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:35.000211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:35.089415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912bbbd-34e9-4f51-9b1d-d9bab90c6751 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:35.093951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:35.094751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:35.095030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:35.095298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] PATH_INFO: `/v3/users/66216ab8e87a4723a9ac450d61d61b4b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:35.096542 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66216ab8e87a4723a9ac450d61d61b4b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:35.098867 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:35.124715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:35.165892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66216ab8e87a4723a9ac450d61d61b4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.166754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66216ab8e87a4723a9ac450d61d61b4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.178022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66216ab8e87a4723a9ac450d61d61b4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.192042 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66216ab8e87a4723a9ac450d61d61b4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.206803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66216ab8e87a4723a9ac450d61d61b4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.214197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1300/2604] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:34 2022] DELETE /identity/v3/users/66216ab8e87a4723a9ac450d61d61b4b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:35.218778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:35.219269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:35.219373 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:35.239885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:35.241128 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:35.286590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:35.331756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1305/2605] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:35.342991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:35.345300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:35.407434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9aa4b-ca8d-43ec-8f74-3f01cf7dc609 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:35.411054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:35.411665 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:35.412149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:35.412424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:35.421779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:35.423757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:35.435305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1301/2606] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:35.722616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:35.724471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:35.741893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624caca-9306-4f0f-85d0-6d2cb4546ad6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:35.746865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:35.747527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:35.747809 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:35.748203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] PATH_INFO: `/v3/projects/cc4d2fba0a7b4e74bb368e0f75548f84` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:35.749334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc4d2fba0a7b4e74bb368e0f75548f84)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:35.751268 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:35.799642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc4d2fba0a7b4e74bb368e0f75548f84', 'request_id': 'req-bef6ff41-f278-41a1-bc4c-a994367b19e3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.805119 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc4d2fba0a7b4e74bb368e0f75548f84', 'request_id': 'req-bef6ff41-f278-41a1-bc4c-a994367b19e3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.812012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc4d2fba0a7b4e74bb368e0f75548f84', 'request_id': 'req-bef6ff41-f278-41a1-bc4c-a994367b19e3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.823387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] The token cache is being invalidate because project cc4d2fba0a7b4e74bb368e0f75548f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:35.823677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.824348 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.825864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1306/2607] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:35 2022] DELETE /identity/v3/projects/cc4d2fba0a7b4e74bb368e0f75548f84 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:35.829954 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:35.839581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614d84-05cc-4bef-870c-9665c0c36744 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:35.901221 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:35.901879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:35.902161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:35.902435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] PATH_INFO: `/v3/users/2559cd3819314157a6966f570f0cecca` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:35.903610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2559cd3819314157a6966f570f0cecca)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:35.905690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:35.943581 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:35.964037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2559cd3819314157a6966f570f0cecca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.964789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2559cd3819314157a6966f570f0cecca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.973753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2559cd3819314157a6966f570f0cecca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.981233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2559cd3819314157a6966f570f0cecca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:35.996308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dbc74959-d7d3-4164-9d49-3756752ab3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2559cd3819314157a6966f570f0cecca'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.002539 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1302/2608] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:35 2022] DELETE /identity/v3/users/2559cd3819314157a6966f570f0cecca => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:36.286168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.287708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:36.302632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6ff41-f278-41a1-bc4c-a994367b19e3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.380937 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.381450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.381737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.382020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] PATH_INFO: `/v3/projects/7b404265822a420a9356a10f9469bd33` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.383395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b404265822a420a9356a10f9469bd33)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.385368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.459193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b404265822a420a9356a10f9469bd33', 'request_id': 'req-ca67a6d4-aae9-4780-8e1d-bf957083a175'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.470006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b404265822a420a9356a10f9469bd33', 'request_id': 'req-ca67a6d4-aae9-4780-8e1d-bf957083a175'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.472688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:36.487740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b404265822a420a9356a10f9469bd33', 'request_id': 'req-ca67a6d4-aae9-4780-8e1d-bf957083a175'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.513151 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] The token cache is being invalidate because project 7b404265822a420a9356a10f9469bd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:36.513411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.514014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ca67a6d4-aae9-4780-8e1d-bf957083a175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:36.516237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1307/2609] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:36 2022] DELETE /identity/v3/projects/7b404265822a420a9356a10f9469bd33 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:36.521648 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.522073 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.522422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.551331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:36.552579 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:36.624360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1303/2610] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:36.627702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.628020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.628306 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.650394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:36.651801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:36.660945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1308/2611] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:36.665544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.675932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d43698-d22f-42a7-98ad-08573e6d74bc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.680712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.681324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.681627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.681909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.682623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:36.682981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.685000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.698956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1304/2612] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:36.706645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.716735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004842-dc6d-43e6-bfa9-51baa96574bf None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.721670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.722298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.722606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.722884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.723565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.725535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.755806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1309/2613] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:36 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:36.760201 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.771355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50c06-eaa2-4925-af82-9c4d7aa6dcce admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.843774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.844459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.844740 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.845014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.845685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.847601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.871101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1305/2614] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:36 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:36.875304 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.886995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e73f7-57fd-4c94-ae1d-817e6d0f0579 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.892067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.894451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.894451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.894451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.894451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.895599 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.907242 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1310/2615] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:36.911419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.921623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca029-d573-4f2b-962b-de45555acf7e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.927101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.927734 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.928046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.928330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.929034 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.930653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0050-78c4-4717-b286-8a1ae65b69d2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:36.945735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1306/2616] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:36.950112 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:36.960799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9c4a-47aa-4d6f-b178-274beb58315e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:36.966201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:36.967081 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:36.967431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:36.967775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] PATH_INFO: `/v3/projects/9a6eca30639b46feaf071f99392551ce/users/cc0518a60cbd4d29aac22a463330711c/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:36.986125 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6eca30639b46feaf071f99392551ce, user_id=cc0518a60cbd4d29aac22a463330711c, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:36.988973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:37.007435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1311/2617] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:36 2022] PUT /identity/v3/projects/9a6eca30639b46feaf071f99392551ce/users/cc0518a60cbd4d29aac22a463330711c/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:37.017590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:37.018130 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:37.018247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:37.061981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] MFA Rules not processed for user `cc0518a60cbd4d29aac22a463330711c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:37.063202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:37.068002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:37.132781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1307/2618] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:37.145051 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:37.147950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:37.207451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3343-29af-46b0-b4b1-d4a5f1287d59 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:37.211542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:37.212144 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:37.212426 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:37.212701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:37.222239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:37.223754 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b92877-c659-4019-a9f5-e6fbc3f06a4d service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:37.233203 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1312/2619] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:37.256404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:37.331518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb6ded-7abc-4fa3-844f-3a95ec883acf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:37.336994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:37.337908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:37.338189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:37.339046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] PATH_INFO: `/v3/users/cc0518a60cbd4d29aac22a463330711c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:37.339884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0518a60cbd4d29aac22a463330711c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:37.341921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:37.409695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0518a60cbd4d29aac22a463330711c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:37.410790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0518a60cbd4d29aac22a463330711c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:37.424968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0518a60cbd4d29aac22a463330711c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:37.443163 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0518a60cbd4d29aac22a463330711c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:37.469049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0518a60cbd4d29aac22a463330711c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:37.476154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1308/2620] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:37 2022] DELETE /identity/v3/users/cc0518a60cbd4d29aac22a463330711c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:37.480512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:37.481013 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:37.481114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:37.501392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:37.502677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:37.594050 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1313/2621] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:37.607535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:37.609549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:37.658292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:37.712354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f27c8-a71b-4c62-b103-8dba7c716dd7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:37.717746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:37.718315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:37.719088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:37.719951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:37.731777 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:37.733409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f4272-7318-45bb-9d95-1a4e757c30dd service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:37.745879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1309/2622] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:38.196604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.198548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:38.212729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5105ccc-526a-47f0-9340-5f7878a6f5fd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.218139 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.218811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.219145 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.219407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] PATH_INFO: `/v3/projects/9a6eca30639b46feaf071f99392551ce` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.220690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6eca30639b46feaf071f99392551ce)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.222697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.251450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:38.279350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6eca30639b46feaf071f99392551ce', 'request_id': 'req-ccbfaa84-2a43-476d-826f-b27037c14abd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:38.289992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6eca30639b46feaf071f99392551ce', 'request_id': 'req-ccbfaa84-2a43-476d-826f-b27037c14abd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:38.299692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6eca30639b46feaf071f99392551ce', 'request_id': 'req-ccbfaa84-2a43-476d-826f-b27037c14abd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:38.316410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] The token cache is being invalidate because project 9a6eca30639b46feaf071f99392551ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:38.316768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:38.317470 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ccbfaa84-2a43-476d-826f-b27037c14abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:38.323290 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1314/2623] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:38 2022] DELETE /identity/v3/projects/9a6eca30639b46feaf071f99392551ce => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:38.326814 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.328413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.329037 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.366711 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:38.368691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:38.440449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1310/2624] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:38.448360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.448700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.448917 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.470565 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:38.471663 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:38.487246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1315/2625] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:38.490621 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.499171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925385f-e49a-41eb-9580-3a5d21f24479 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.504719 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.505345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.505623 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.505895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.506645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:38.506934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.508601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.526237 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1311/2626] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:38.530389 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.540845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb53ee3-1a24-4dd7-9a50-a8b5e6954351 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.545741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.546407 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.546714 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.546983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.548255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.550180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.581759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1316/2627] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:38 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:38.585717 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.597101 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec215ee-3f31-49b4-9121-bf52ecd01445 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.661565 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.662191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.662478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.662754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.663501 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.665459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.691442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1312/2628] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:38 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:38.697534 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.709959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d2aa5-f30e-45d8-abab-154903ef25ce admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.717277 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.718527 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.719103 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.720222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.721324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.723365 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.745177 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.746792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1317/2629] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:38.759481 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66814-9ea9-4e82-a92d-721e07531018 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.765450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.765450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.765830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.766147 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.767124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.768991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbaeff-c285-4820-8cec-bb799affe023 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.783994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1313/2630] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:38.786695 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:38.800211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0fee-e54d-4291-9657-4a752f57cc8b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:38.805731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:38.806416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.806698 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.806979 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] PATH_INFO: `/v3/projects/1bbdeffcaea54aeca8935e66512fc64b/users/2be3954743364754a6de31983bdcfcc5/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.823997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbdeffcaea54aeca8935e66512fc64b, user_id=2be3954743364754a6de31983bdcfcc5, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:38.826679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:38.857438 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1318/2631] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:38 2022] PUT /identity/v3/projects/1bbdeffcaea54aeca8935e66512fc64b/users/2be3954743364754a6de31983bdcfcc5/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:38.868432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:38.868708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:38.869036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:38.905192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] MFA Rules not processed for user `2be3954743364754a6de31983bdcfcc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:38.906465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:38.937953 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:39.024716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1314/2632] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:39.037087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:39.038859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:39.115293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604c21-e0e8-488b-8938-b99bebeefe66 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:39.119555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:39.120334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:39.120717 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:39.120889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:39.130451 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:39.132060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:39.143276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1319/2633] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:46.048121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:46.050498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:46.184616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a2714-75b9-4f0e-a153-90130c0e59c4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:46.192736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:46.193392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:46.193728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:46.194005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:46.196107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:46.209684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:46.211367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:46.225194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1315/2634] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:46.873266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:46.875901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:46.956502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:46.989898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba4bb-5fde-470f-9425-21e6db1bba26 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:46.998276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:46.998973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:46.999263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:46.999551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] PATH_INFO: `/v3/users/f7c451926101491caa1df6e7cabe118a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:47.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c451926101491caa1df6e7cabe118a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:47.006473 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:47.081502 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c451926101491caa1df6e7cabe118a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.094357 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c451926101491caa1df6e7cabe118a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.117272 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c451926101491caa1df6e7cabe118a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.126079 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c451926101491caa1df6e7cabe118a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.126824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c451926101491caa1df6e7cabe118a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.133676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1320/2635] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:46 2022] DELETE /identity/v3/users/f7c451926101491caa1df6e7cabe118a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:47.336891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:47.348244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26c3b7-d788-423b-8cbe-5d29b3e8ed36 service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:47.410778 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:47.411432 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:47.411704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:47.412008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] PATH_INFO: `/v3/projects/d4fad8f265cd458b8557cf0d21adfa1f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:47.413195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4fad8f265cd458b8557cf0d21adfa1f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:47.415106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:47.458600 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:47.470290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4fad8f265cd458b8557cf0d21adfa1f', 'request_id': 'req-b4596989-28db-4945-9325-570a4a62b35a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.477749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4fad8f265cd458b8557cf0d21adfa1f', 'request_id': 'req-b4596989-28db-4945-9325-570a4a62b35a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.488030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4fad8f265cd458b8557cf0d21adfa1f', 'request_id': 'req-b4596989-28db-4945-9325-570a4a62b35a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.503558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] The token cache is being invalidate because project d4fad8f265cd458b8557cf0d21adfa1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:47.503859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.504549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.506152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1316/2636] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:47 2022] DELETE /identity/v3/projects/d4fad8f265cd458b8557cf0d21adfa1f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:47.510282 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:47.512042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:47.526957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923b9b9-6bcf-4ad2-80ac-4120bb450a95 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:47.580846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:47.582679 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:47.611737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:47.612737 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:47.613105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:47.613307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] PATH_INFO: `/v3/users/9e5240923c8746b082b2e22af898541c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:47.614965 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5240923c8746b082b2e22af898541c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:47.616888 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:47.689182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5240923c8746b082b2e22af898541c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.698213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5240923c8746b082b2e22af898541c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.699252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596989-28db-4945-9325-570a4a62b35a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:47.707281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:47.707281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:47.707281 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:47.707967 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] PATH_INFO: `/v3/users/2be3954743364754a6de31983bdcfcc5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:47.708811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be3954743364754a6de31983bdcfcc5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:47.711133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:47.722837 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:47.725461 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5240923c8746b082b2e22af898541c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.746385 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5240923c8746b082b2e22af898541c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.746672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9b055d6c-814e-4608-b2c3-3c0bdfe6dc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5240923c8746b082b2e22af898541c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.762924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1321/2637] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:47 2022] DELETE /identity/v3/users/9e5240923c8746b082b2e22af898541c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:47.825200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be3954743364754a6de31983bdcfcc5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.831056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be3954743364754a6de31983bdcfcc5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.841259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be3954743364754a6de31983bdcfcc5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.859386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be3954743364754a6de31983bdcfcc5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.884087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be3954743364754a6de31983bdcfcc5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:47.892091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1317/2638] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:36:47 2022] DELETE /identity/v3/users/2be3954743364754a6de31983bdcfcc5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:47.896914 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:47.897483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:47.897483 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:47.919869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:47.921254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:47.994107 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:47.995739 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.032750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1322/2639] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:48.043775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.045335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.073026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:48.115544 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede455ae-4ced-4175-bdea-1283278a4307 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.122925 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.124616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.125972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.126961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] PATH_INFO: `/v3/projects/c643ea4cc37b45eea4346404772e46a7` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.130142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c643ea4cc37b45eea4346404772e46a7)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.133067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.145064 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356a8f-b69e-464b-a3df-18acecfc7c1a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.149246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.149890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.150646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.150646 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.163395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.164793 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea64b85-582f-4e50-b474-1ade9e785ed5 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.178824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1323/2640] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:48.209840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c643ea4cc37b45eea4346404772e46a7', 'request_id': 'req-61dd1b4b-7508-4eed-9faf-7bd23fad6411'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.218133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c643ea4cc37b45eea4346404772e46a7', 'request_id': 'req-61dd1b4b-7508-4eed-9faf-7bd23fad6411'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.229016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c643ea4cc37b45eea4346404772e46a7', 'request_id': 'req-61dd1b4b-7508-4eed-9faf-7bd23fad6411'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.250865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] The token cache is being invalidate because project c643ea4cc37b45eea4346404772e46a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:48.251626 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.252810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-61dd1b4b-7508-4eed-9faf-7bd23fad6411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.258383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1318/2641] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:47 2022] DELETE /identity/v3/projects/c643ea4cc37b45eea4346404772e46a7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:48.260414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.260801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.261088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.288999 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:48.290328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.402487 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1324/2642] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:48.417846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.417846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.417846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.447222 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:48.449672 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.465697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1319/2643] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:48.467057 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.479107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78177b2-d9e4-41eb-94d5-c3ca9614dacc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.487239 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.488671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.488671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.488671 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.489280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:48.489610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.491538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.500891 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:48.512866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1325/2644] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:36:48.514276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.535937 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdd475-77ea-40a8-a729-89335b9435c5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.544723 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.545691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.546054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.546456 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.547861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.550492 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.561486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.568033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.601363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3acabd-d98c-4662-81cf-b9f1a85a962f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.607957 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1320/2645] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:48.610097 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.692999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b9fa-faa5-4a34-aab5-f50124e81a42 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.698617 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.699534 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.699656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.700064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.700400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.700778 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.701188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.701306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.701686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] PATH_INFO: `/v3/projects/1bbdeffcaea54aeca8935e66512fc64b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.703637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.703867 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbdeffcaea54aeca8935e66512fc64b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.708045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.740005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1321/2646] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:48.746129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.760619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a15f7-8008-40a8-8953-31bfbf238029 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.769129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.770369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.770879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.771345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.772667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.775519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.796148 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1322/2647] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:48.798592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbdeffcaea54aeca8935e66512fc64b', 'request_id': 'req-60b52338-5fb9-4a62-b47f-38cedab3af1b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.801304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.811568 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbdeffcaea54aeca8935e66512fc64b', 'request_id': 'req-60b52338-5fb9-4a62-b47f-38cedab3af1b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.820304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d817-4b9c-4cc6-8721-c05638f16884 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.823293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbdeffcaea54aeca8935e66512fc64b', 'request_id': 'req-60b52338-5fb9-4a62-b47f-38cedab3af1b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.846511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:48.848884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] The token cache is being invalidate because project 1bbdeffcaea54aeca8935e66512fc64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:36:48.849284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.854250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-60b52338-5fb9-4a62-b47f-38cedab3af1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:36:48.854250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1326/2648] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:36:48 2022] DELETE /identity/v3/projects/1bbdeffcaea54aeca8935e66512fc64b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:48.857230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c198-0376-493d-9965-58f597c97fd0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.857789 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c198-0376-493d-9965-58f597c97fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.857982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c198-0376-493d-9965-58f597c97fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.883967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a329c198-0376-493d-9965-58f597c97fd0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:48.885750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a329c198-0376-493d-9965-58f597c97fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.922479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.923330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.923613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.923944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.924666 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:48.926356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:48.938804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1323/2649] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:48.942769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:48.944866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:48.965792 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a85c2c-c759-4f22-99bb-beda1d09d433 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:48.973335 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:48.974141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.974435 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.974716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] PATH_INFO: `/v3/projects/fff7d082a71e42628eb50be48a01e702/users/88cf2f82b09c48dc91db358f7d0f0b31/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:48.977065 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1327/2650] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:48.989274 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:48.990071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:48.990499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.017150 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff7d082a71e42628eb50be48a01e702, user_id=88cf2f82b09c48dc91db358f7d0f0b31, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.018799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:49.019994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8492be8-1861-46da-91e8-035b7661f086 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.020233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:49.030493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1328/2651] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:49.037021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.045482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1324/2652] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:48 2022] PUT /identity/v3/projects/fff7d082a71e42628eb50be48a01e702/users/88cf2f82b09c48dc91db358f7d0f0b31/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:49.049436 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d42770-4cb0-44e5-b2a2-25b2cfdc76b8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.085707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:49.121485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.121773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.122053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.152409 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.153192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.153539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.153746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.154862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:36:49.155069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] MFA Rules not processed for user `88cf2f82b09c48dc91db358f7d0f0b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:49.155241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.155907 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:49.157538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.173682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1329/2653] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 15:36:49.178020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.191363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b2e05-43c2-4a6f-91d7-b1056de849ee admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.197553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.198535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.199382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.199545 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.200684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.204897 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.244149 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1330/2654] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:49.248201 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.253445 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1325/2655] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:49.258123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.270939 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e26ede-2a99-46d7-a4bc-46f315ec696a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.380518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf8c8-f457-4f79-89ec-8bf8b7a710da admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.388433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.389554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.389991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.391148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.392672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.396287 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.400550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.401722 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.402447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.402920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.404090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.407857 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.436673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1331/2656] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:49.441224 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.444428 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1326/2657] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:49.448658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.452188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191b2a3-854c-4f58-ad8e-cf0b3393069d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.461974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:49.552751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.553818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.554261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.555039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.555955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.558143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.558836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144fe01-e60c-40bc-879a-d3582358ada2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.564835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.565059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.565330 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.565633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.566311 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.568512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.574958 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1332/2658] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:49.585126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.598448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db655f23-9508-40eb-a090-8814366f06c9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.605583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1327/2659] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:36:49.605992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.606713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.606996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.607280 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.608118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.609907 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.611040 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.624019 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1333/2660] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:49.626887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61f95-69bf-444b-9274-655573fb1741 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.627948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.636633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.638013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.638473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.638934 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.639040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2074e-da63-4e61-be51-ae6a0cfa46c9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.640254 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.644395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.649367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.650578 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.651080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.651538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] PATH_INFO: `/v3/projects/1f420f7f5f5947088f23da2dab9f0fd7/users/37a42563c3d34a689c3cec947d649fc1/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.657826 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1328/2661] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:49.658062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:49.663566 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.675551 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f420f7f5f5947088f23da2dab9f0fd7, user_id=37a42563c3d34a689c3cec947d649fc1, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.678890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bd603-94ba-4924-a764-808a544d92cc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.679994 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaca46-cd7a-4696-983e-5c9d91064be0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.688979 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.689819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.690204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.690409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] PATH_INFO: `/v3/projects/57c2531fe60e43caa12167b5ccfad3f2/users/e5f351d295864c8cbb511e5181500e4c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.697237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1334/2662] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:49 2022] PUT /identity/v3/projects/1f420f7f5f5947088f23da2dab9f0fd7/users/37a42563c3d34a689c3cec947d649fc1/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:49.702375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.702680 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.702799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.709385 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c2531fe60e43caa12167b5ccfad3f2, user_id=e5f351d295864c8cbb511e5181500e4c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.713305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.725254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:49.726664 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:49.732694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1329/2663] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:49 2022] PUT /identity/v3/projects/57c2531fe60e43caa12167b5ccfad3f2/users/e5f351d295864c8cbb511e5181500e4c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:49.733697 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.814525 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacb959-d464-42b9-863c-265b9ceb45e9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.817069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1335/2664] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:49.819856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.820624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.821260 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.821898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.822660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.824439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.829549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.832093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:49.837512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1330/2665] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:49.841900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:49.856086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac420194-0932-40e8-bfaf-707c14f7cdcb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.864024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.865416 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.865816 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.866135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] PATH_INFO: `/v3/projects/57c2531fe60e43caa12167b5ccfad3f2/users/e5f351d295864c8cbb511e5181500e4c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.871772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c2531fe60e43caa12167b5ccfad3f2, user_id=e5f351d295864c8cbb511e5181500e4c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:49.874547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40a8f2-3cc6-4914-b8df-b25a39ba06e9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:49.888819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1331/2666] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:49 2022] PUT /identity/v3/projects/57c2531fe60e43caa12167b5ccfad3f2/users/e5f351d295864c8cbb511e5181500e4c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:36:49.899800 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.900096 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.900395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.912863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000ea8f-6d1a-437e-a491-de2d1cba117d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:49.934634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] MFA Rules not processed for user `e5f351d295864c8cbb511e5181500e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:49.936187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:49.990096 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:49.991017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:49.991549 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:49.991866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:49.993046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:50.045535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1332/2667] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:50.056694 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:50.058399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:50.070544 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:50.096366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:50.097949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6b57d-e2eb-4bf5-85c9-7581f3f0b23b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:50.110093 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1336/2668] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:50.111410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:50.111705 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:50.112009 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:50.132904 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] MFA Rules not processed for user `691d2cd293b244398bc9a3a98f1ad928`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:50.133810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8e9f6-0aff-46a1-b8a6-97e0ba366207 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:50.135181 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:50.138715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:50.139447 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:50.140094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:50.140513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:50.226324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1337/2669] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:36:50.229981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:50.231903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c147eb-57a7-4c01-9d9b-80c2cbd23489 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:50.243399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1333/2670] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:36:55.192521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:55.193988 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:55.253898 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08caa65-d682-4b67-8b0a-f556c2de0492 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:55.258246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:55.258865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:55.259204 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:55.259475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:55.271707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:55.273222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:55.284910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1338/2671] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:36:56.705804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:56.706287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:56.706475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:56.740358 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] MFA Rules not processed for user `37a42563c3d34a689c3cec947d649fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:36:56.741912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:36:56.830308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1334/2672] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:36:56.835033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:56.845746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e49296-84f8-4bfb-bb3c-1cf58ff30d14 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:56.850326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:56.850949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:56.851233 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:56.851505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:56.852411 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:56.854325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:56.893178 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1339/2673] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:36:56 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:56.901024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:56.913382 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e841078-0345-46a8-bc1f-214de74c2bf6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:56.927904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:36:57.131021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:57.131580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:57.131909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:57.132153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:57.133021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:57.135024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:57.172360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1335/2674] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:36:56 2022] POST /identity/v3/users => generated 292 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:36:57.174825 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:57.190205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238dc18-a113-47ad-b086-48e3f6ecc128 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:57.211941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:57.212679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:57.212961 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:57.215665 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:57.216772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:57.220619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:57.256101 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1340/2675] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:36:57.257153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:57.271569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a66f-1f92-46fb-96f3-1ce7d5e85779 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:57.277528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:57.278670 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:57.279429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:57.280050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] PATH_INFO: `/v3/projects/e6dc7bd3aaf94f6db124873ade24362e/users/ed49602d429344f48ccb41c11f575c4c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:57.305567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6dc7bd3aaf94f6db124873ade24362e, user_id=ed49602d429344f48ccb41c11f575c4c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:57.309217 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:57.357142 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1336/2676] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:57 2022] PUT /identity/v3/projects/e6dc7bd3aaf94f6db124873ade24362e/users/ed49602d429344f48ccb41c11f575c4c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:36:57.362557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:57.377368 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa3537-3303-434a-b87f-3fbbb9c31cb7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:57.458269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:57.459088 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:57.459434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:57.459776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:57.460944 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:57.462873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:57.480024 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1341/2677] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:36:57.481329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:36:57.495309 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869667-d215-4de2-b9bd-45741e39ef71 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:36:57.501759 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:36:57.503138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:36:57.503704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:36:57.504246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] PATH_INFO: `/v3/projects/e6dc7bd3aaf94f6db124873ade24362e/users/ed49602d429344f48ccb41c11f575c4c/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:36:57.506846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6dc7bd3aaf94f6db124873ade24362e, user_id=ed49602d429344f48ccb41c11f575c4c, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:36:57.509971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c5ac7-cfbc-4071-8f39-9a0f8a01077e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:36:57.525093 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1337/2678] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:36:57 2022] PUT /identity/v3/projects/e6dc7bd3aaf94f6db124873ade24362e/users/ed49602d429344f48ccb41c11f575c4c/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:00.799237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:00.826708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add10b7-4742-4a5c-8509-d3783967880a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:00.871134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:00.922433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:00.923755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:00.923755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:00.924011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] PATH_INFO: `/v3/users/88cf2f82b09c48dc91db358f7d0f0b31` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:00.925574 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88cf2f82b09c48dc91db358f7d0f0b31)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:00.927683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:01.001715 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88cf2f82b09c48dc91db358f7d0f0b31'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:01.012198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88cf2f82b09c48dc91db358f7d0f0b31'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:01.028772 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88cf2f82b09c48dc91db358f7d0f0b31'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:01.037421 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88cf2f82b09c48dc91db358f7d0f0b31'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:01.038369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88cf2f82b09c48dc91db358f7d0f0b31'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:01.046911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1342/2679] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:00 2022] DELETE /identity/v3/users/88cf2f82b09c48dc91db358f7d0f0b31 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:01.052491 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:01.053019 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:01.053689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:01.091919 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:01.093835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:01.187962 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1338/2680] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:37:01.206529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:01.208478 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:01.353218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1cd5c-7cb0-45db-b361-f00c2be5cb10 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:01.362542 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:01.363276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:01.363668 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:01.364190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:01.378921 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:01.380878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:01.394278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1343/2681] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:02.014000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:02.016350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:02.030837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d293ec-a450-4849-a0e0-8f350915d2d6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:02.036304 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:02.036966 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:02.037271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:02.037866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] PATH_INFO: `/v3/projects/fff7d082a71e42628eb50be48a01e702` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:02.039175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff7d082a71e42628eb50be48a01e702)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:02.042361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:02.119546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff7d082a71e42628eb50be48a01e702', 'request_id': 'req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.131178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff7d082a71e42628eb50be48a01e702', 'request_id': 'req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.144616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff7d082a71e42628eb50be48a01e702', 'request_id': 'req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.151230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:02.179300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] The token cache is being invalidate because project fff7d082a71e42628eb50be48a01e702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:02.179951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.180913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6b5bb57c-a249-4e60-8581-b56d99f7cfdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.183917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1339/2682] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:02 2022] DELETE /identity/v3/projects/fff7d082a71e42628eb50be48a01e702 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:02.188684 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:02.204831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fc4d4-c32c-447c-ad2b-0069e3d3bd27 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:02.283464 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:02.299424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:02.300679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:02.302136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:02.302136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] PATH_INFO: `/v3/users/e5f351d295864c8cbb511e5181500e4c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:02.304603 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f351d295864c8cbb511e5181500e4c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:02.307180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:02.372414 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f351d295864c8cbb511e5181500e4c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.384299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f351d295864c8cbb511e5181500e4c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.401567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f351d295864c8cbb511e5181500e4c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.415417 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f351d295864c8cbb511e5181500e4c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.422785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f351d295864c8cbb511e5181500e4c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:02.433569 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1344/2683] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:02 2022] DELETE /identity/v3/users/e5f351d295864c8cbb511e5181500e4c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:02.797228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:02.797547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:02.797824 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:02.830864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] MFA Rules not processed for user `ed49602d429344f48ccb41c11f575c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:02.831784 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:02.934232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1340/2684] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:02.954499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:02.960004 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:03.122515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.125181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:03.144211 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a777a-624c-409a-89af-f4b90fcdd1f4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.163862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.165273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.167087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.168406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.176118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:03.228095 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de429e-9631-41d7-b2f2-24183d12a46c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.233514 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.234190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.234475 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.234751 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] PATH_INFO: `/v3/projects/57c2531fe60e43caa12167b5ccfad3f2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.236298 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c2531fe60e43caa12167b5ccfad3f2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.240045 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.286948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:03.304241 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c2531fe60e43caa12167b5ccfad3f2', 'request_id': 'req-c675470b-96a0-434f-87df-b91854693f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:03.312244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c2531fe60e43caa12167b5ccfad3f2', 'request_id': 'req-c675470b-96a0-434f-87df-b91854693f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:03.324413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c2531fe60e43caa12167b5ccfad3f2', 'request_id': 'req-c675470b-96a0-434f-87df-b91854693f52'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:03.327799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.339415 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a925d-939f-43db-af70-468cf4cb641c service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.343728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] The token cache is being invalidate because project 57c2531fe60e43caa12167b5ccfad3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:03.344090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:03.345714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c675470b-96a0-434f-87df-b91854693f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:03.347008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1341/2685] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:03 2022] DELETE /identity/v3/projects/57c2531fe60e43caa12167b5ccfad3f2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:37:03.356668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.357178 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.357407 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.374497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1345/2686] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:03.384986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:03.400021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:03.402786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:03.551154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1342/2687] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:03.563718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.564105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.564309 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.600089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:03.601356 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:03.613492 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1346/2688] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:03.617558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.629453 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7fa9-a87c-443d-8bbd-287b4570bab0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.636094 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.636931 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.637197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.637478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.638198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:37:03.638505 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.640900 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.661805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1343/2689] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:37:03.663276 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.680678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51bd22-54fa-44d7-900d-a8f33436d4f0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.702808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.703449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.703762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.704171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.704848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.708613 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.745639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1347/2690] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:37:03 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:03.750632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.761686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35c574-a47c-4478-98b0-9ef51264da36 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.839224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.840062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.840351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.840629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.841307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.843276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.872547 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1344/2691] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:37:03 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:03.877548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.895952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176a757-affe-4e9c-83b2-3675345cd072 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.909129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.910520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.911247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.911781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.913036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.915197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.933078 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1348/2692] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:37:03.935000 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:03.957114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac100531-8c10-4f16-8d21-9dbe8365b640 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:03.969898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:03.971317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:03.971769 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:03.972103 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:03.972834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:03.974510 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7496-b230-4983-af1f-9e328878bbca admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:03.987895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1345/2693] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:03.991335 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.002315 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5386a4-2541-416d-bc07-c947602fd782 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.008413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.009129 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.009416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.010202 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] PATH_INFO: `/v3/projects/c21b9286eb2945dba27f458fd247c158/users/d84166c229854c728fe2ba2d1e5c1597/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.032175 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21b9286eb2945dba27f458fd247c158, user_id=d84166c229854c728fe2ba2d1e5c1597, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.037031 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.068986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1349/2694] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:03 2022] PUT /identity/v3/projects/c21b9286eb2945dba27f458fd247c158/users/d84166c229854c728fe2ba2d1e5c1597/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:04.082194 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.082485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.082762 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.124336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] MFA Rules not processed for user `d84166c229854c728fe2ba2d1e5c1597`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:04.125764 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:04.223951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1346/2695] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:04.228922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.312107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d2d8e-4ab2-4b27-a407-3c605e6744be admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.317763 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.318672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.318968 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.319232 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.320133 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.322223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.356660 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1350/2696] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:04.360973 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.368023 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:04.377010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb09fb0-32ae-4321-bbaa-c5c3bd54f02b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.457496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.458137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.458414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.458683 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.459356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.461315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.484609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1347/2697] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:04.488687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.500742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70372369-b1c8-4edb-ac20-2db74210313e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.505519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.506147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.506428 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.506702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.507394 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.509067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.521258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1351/2698] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:04.525345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.536381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06d31b-adc4-435b-a2af-6a4071b46bde admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.541439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.542041 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.542488 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.542609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.543271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.544895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f86df6-8bbe-4f77-9abe-4c73fa622ca3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.556243 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1348/2699] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:04.560728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.575758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e958-2011-406d-ac2c-ee32118b55b7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.585960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.587023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.587023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.587530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] PATH_INFO: `/v3/projects/3920da7b911c4ea19ce290251330a426/users/106bf4fbcf544cb9b3453e2f5916ccff/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.607452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3920da7b911c4ea19ce290251330a426, user_id=106bf4fbcf544cb9b3453e2f5916ccff, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.610768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c343ef-406c-47cd-a1eb-534cf40b64d7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.626329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1352/2700] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:04 2022] PUT /identity/v3/projects/3920da7b911c4ea19ce290251330a426/users/106bf4fbcf544cb9b3453e2f5916ccff/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:04.637629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29301-3510-4292-adab-355eada02356 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.637887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29301-3510-4292-adab-355eada02356 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.638226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29301-3510-4292-adab-355eada02356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.674981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e4b29301-3510-4292-adab-355eada02356 None None] MFA Rules not processed for user `106bf4fbcf544cb9b3453e2f5916ccff`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:04.676854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e4b29301-3510-4292-adab-355eada02356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:04.724254 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.724547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.724829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.746562 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] MFA Rules not processed for user `37a42563c3d34a689c3cec947d649fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:04.748347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:04.763129 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1349/2701] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:04.767869 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b29301-3510-4292-adab-355eada02356 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.775694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:04.821581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1353/2702] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:37:04.849204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b29301-3510-4292-adab-355eada02356 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:04.853789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:04.854423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:04.854838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:04.855119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:04.855796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:04.857970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:04.878087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:04.897005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1350/2703] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:04.898122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:04.912162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93303c2a-fbfa-433b-a22d-48274be3a2d4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.006522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.007785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.008113 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.012831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.013857 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.016036 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.060819 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:05.068761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1354/2704] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:37:04 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:37:05.076614 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f182f2-04d2-42ae-84bf-697231ecf54f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.082989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.083658 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.083982 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.084257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.085716 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.087582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.101556 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1351/2705] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:37:05.106184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:05.119941 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb109-5459-44c8-9254-0232ec5633db admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.137964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.138190 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.138299 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.138557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] PATH_INFO: `/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e/users/467489166c7c4fd19c1adfb730ad4bf7/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.168089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d9844bc8ed4d9aad033289d85bbf7e, user_id=467489166c7c4fd19c1adfb730ad4bf7, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.172186 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.191213 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1355/2706] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:05 2022] PUT /identity/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e/users/467489166c7c4fd19c1adfb730ad4bf7/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:05.195286 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:05.210938 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4af5-a9b5-4f0a-a6d2-93cf9f87bc22 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.295486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.296307 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.296442 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.296713 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.297412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.299028 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e5b95-25ea-4e8e-9ea4-1607423a4393 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.311083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1352/2707] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:05.315344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:05.327759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11a1b-5129-42a3-950e-9c849b7f3620 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.332839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.333512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.333802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.334080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] PATH_INFO: `/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e/users/467489166c7c4fd19c1adfb730ad4bf7/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.338561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d9844bc8ed4d9aad033289d85bbf7e, user_id=467489166c7c4fd19c1adfb730ad4bf7, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.341289 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.353649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1356/2708] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:05 2022] PUT /identity/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e/users/467489166c7c4fd19c1adfb730ad4bf7/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:05.364835 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.365119 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.365396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.396355 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] MFA Rules not processed for user `467489166c7c4fd19c1adfb730ad4bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:05.397592 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:05.496930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1353/2709] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:05.510363 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:05.512167 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:05.583732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56f86e-d9e8-47e4-a394-c67f03edc30e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:05.587853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:05.588517 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:05.588795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:05.589066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:05.636131 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:05.663352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:05.664830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:05.676419 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1357/2710] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:07.280111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:07.290794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2af9-3a70-484c-bcba-080639a78221 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:07.294654 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:07.295793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:07.296114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:07.296408 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:07.386909 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:07.390383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:07.415127 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1354/2711] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:08.336525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:08.348207 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6ca3-3c53-4031-a64c-849e006df6bb service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:08.356785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:08.357718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:08.358011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:08.358336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:08.370034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:08.371538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac73f8-21f0-44fa-bc24-652fab87c911 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:08.400069 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1358/2712] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:12.422913 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:12.429044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:12.490484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7421364-5162-422e-9992-21e988f7eee3 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:12.495080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:12.495744 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:12.496038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:12.496315 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] PATH_INFO: `/v3/users/d84166c229854c728fe2ba2d1e5c1597` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:12.497489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84166c229854c728fe2ba2d1e5c1597)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:12.499457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:12.548989 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84166c229854c728fe2ba2d1e5c1597'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:12.549661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84166c229854c728fe2ba2d1e5c1597'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:12.556801 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84166c229854c728fe2ba2d1e5c1597'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:12.568176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84166c229854c728fe2ba2d1e5c1597'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:12.580548 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84166c229854c728fe2ba2d1e5c1597'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:12.587673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1355/2713] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:12 2022] DELETE /identity/v3/users/d84166c229854c728fe2ba2d1e5c1597 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:12.591106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:12.591581 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:12.591679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:12.619922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:12.621132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:12.691639 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1359/2714] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:12.710528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:12.712917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:12.784294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588531bd-2511-4e93-a22f-8fb74eac9eb7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:12.787908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:12.788701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:12.788977 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:12.789249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:12.800361 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:12.802460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:12.814022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1356/2715] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:13.141025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:13.142566 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:13.162742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef7e3-9bc1-40f1-bc7e-3457c52a7a01 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:13.169333 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:13.170193 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:13.170649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:13.171042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] PATH_INFO: `/v3/projects/c21b9286eb2945dba27f458fd247c158` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:13.172431 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21b9286eb2945dba27f458fd247c158)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:13.174347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:13.233723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21b9286eb2945dba27f458fd247c158', 'request_id': 'req-22a99e20-f57f-433d-978b-439a742042ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.239307 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21b9286eb2945dba27f458fd247c158', 'request_id': 'req-22a99e20-f57f-433d-978b-439a742042ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.246740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21b9286eb2945dba27f458fd247c158', 'request_id': 'req-22a99e20-f57f-433d-978b-439a742042ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.258842 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] The token cache is being invalidate because project c21b9286eb2945dba27f458fd247c158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:13.259136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.259866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.261498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1360/2716] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:13 2022] DELETE /identity/v3/projects/c21b9286eb2945dba27f458fd247c158 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:13.269146 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:13.279399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf00657-808c-4548-b040-d9ddddc85dab service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:13.304484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:13.342079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:13.342660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:13.342947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:13.343220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] PATH_INFO: `/v3/users/106bf4fbcf544cb9b3453e2f5916ccff` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:13.344422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=106bf4fbcf544cb9b3453e2f5916ccff)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:13.346329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:13.407807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106bf4fbcf544cb9b3453e2f5916ccff'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.408503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106bf4fbcf544cb9b3453e2f5916ccff'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.416757 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106bf4fbcf544cb9b3453e2f5916ccff'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.424172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106bf4fbcf544cb9b3453e2f5916ccff'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.437350 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106bf4fbcf544cb9b3453e2f5916ccff'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.444373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1357/2717] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:13 2022] DELETE /identity/v3/users/106bf4fbcf544cb9b3453e2f5916ccff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:13.755150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:13.762871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:13.783707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99e20-f57f-433d-978b-439a742042ed admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:13.832446 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:13.847294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:13.847981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:13.848265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:13.848591 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] PATH_INFO: `/v3/projects/3920da7b911c4ea19ce290251330a426` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:13.849704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3920da7b911c4ea19ce290251330a426)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:13.851645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:13.911982 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3920da7b911c4ea19ce290251330a426', 'request_id': 'req-36f6cc50-9175-4dbf-bc7c-65251e701e67'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.930042 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3920da7b911c4ea19ce290251330a426', 'request_id': 'req-36f6cc50-9175-4dbf-bc7c-65251e701e67'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.939718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3920da7b911c4ea19ce290251330a426', 'request_id': 'req-36f6cc50-9175-4dbf-bc7c-65251e701e67'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.951526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] The token cache is being invalidate because project 3920da7b911c4ea19ce290251330a426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:13.951818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.952853 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:13.955550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1361/2718] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:13 2022] DELETE /identity/v3/projects/3920da7b911c4ea19ce290251330a426 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:13.961090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:13.962847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:13.976372 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0881f-2ca3-444e-b72f-c7eb78a780ad admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.038300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.038951 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.039240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.039543 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] PATH_INFO: `/v3/users/467489166c7c4fd19c1adfb730ad4bf7` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.040868 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=467489166c7c4fd19c1adfb730ad4bf7)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:14.042923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:14.096619 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467489166c7c4fd19c1adfb730ad4bf7'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.097328 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467489166c7c4fd19c1adfb730ad4bf7'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.099267 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:14.106012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467489166c7c4fd19c1adfb730ad4bf7'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.115144 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467489166c7c4fd19c1adfb730ad4bf7'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.127051 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f3cfedcc-fc8f-47eb-8993-692f155c05f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467489166c7c4fd19c1adfb730ad4bf7'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.137003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1358/2719] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:13 2022] DELETE /identity/v3/users/467489166c7c4fd19c1adfb730ad4bf7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:14.436360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:14.437799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:14.450936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cc50-9175-4dbf-bc7c-65251e701e67 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.521940 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.522753 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.523035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.523310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] PATH_INFO: `/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.524605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d9844bc8ed4d9aad033289d85bbf7e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:14.526538 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:14.588032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d9844bc8ed4d9aad033289d85bbf7e', 'request_id': 'req-9426cc59-6b9f-4947-918b-80b92d5e0f34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.595748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d9844bc8ed4d9aad033289d85bbf7e', 'request_id': 'req-9426cc59-6b9f-4947-918b-80b92d5e0f34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.604862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d9844bc8ed4d9aad033289d85bbf7e', 'request_id': 'req-9426cc59-6b9f-4947-918b-80b92d5e0f34'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.620586 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] The token cache is being invalidate because project 08d9844bc8ed4d9aad033289d85bbf7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:14.620928 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.621655 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9426cc59-6b9f-4947-918b-80b92d5e0f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:14.623180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1362/2720] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:14 2022] DELETE /identity/v3/projects/08d9844bc8ed4d9aad033289d85bbf7e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:14.628277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.628795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.628795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.649549 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:14.650774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:14.720643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1359/2721] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:14.728702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.728989 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.729263 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.743811 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:14.751901 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:14.757633 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:14.766134 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1363/2722] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:14.770687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:14.781265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7763a1-5ef6-4c64-921a-2d287560b7d5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.787159 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.788263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.788610 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.789014 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.789915 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:37:14.790226 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:14.792980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:14.807580 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1360/2723] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:37:14.812535 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:14.822371 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a720c9-0649-433d-998d-6a87ae999c30 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.827623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.828267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.828547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.828820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.829572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:14.831441 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:14.860210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1364/2724] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:37:14 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:14.865099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:14.876198 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4df7e-35a5-4566-a0b9-fcde7f5d6076 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.937894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.938518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.938805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.939075 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.939890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:14.941812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:14.971477 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1361/2725] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:37:14 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:14.975806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:14.990556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf300e8e-af26-449e-86e2-496ab6686bd6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:14.995862 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:14.996482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:14.996769 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:14.997058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:14.997771 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:15.000269 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:15.011132 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1365/2726] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:15.015811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:15.026667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5460ed-8ddf-49e5-8cd3-dff150f07a4b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:15.031522 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:15.032174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:15.032454 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:15.032730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:15.033423 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:15.035059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7af126-f2f5-4d90-aab2-afd61aef9ec5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:15.047124 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1362/2727] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:37:15.051954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:15.064967 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86190034-e710-4524-9c12-627be77028c4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:15.070601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:15.071305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:15.071686 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:15.072074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] PATH_INFO: `/v3/projects/a3928eed5c394bc099903b4bc6f7ed20/users/197918379fad428893add837f3024678/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:15.087706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3928eed5c394bc099903b4bc6f7ed20, user_id=197918379fad428893add837f3024678, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:15.090406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:15.107652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1366/2728] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:15 2022] PUT /identity/v3/projects/a3928eed5c394bc099903b4bc6f7ed20/users/197918379fad428893add837f3024678/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:37:15.119602 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:15.119811 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:15.120128 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:15.152590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] MFA Rules not processed for user `197918379fad428893add837f3024678`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:15.153805 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:15.173134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:15.221141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1363/2729] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:15.233430 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:15.235157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:15.295071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1036523-d430-4cc3-9654-ed10179b6cb0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:15.299352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:15.300015 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:15.300322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:15.300600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:15.311546 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:15.313070 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:15.322422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1367/2730] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:25.218728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=18 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:25.318281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:42.265464 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:42.267086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:42.333205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0ccbf-4e19-41ed-9f78-dd63324680b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:42.336902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:42.337531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:42.337806 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:42.338079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:42.339862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:42.408336 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:42.409768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d97a6-e98e-41a3-a5d5-46f39ec7b241 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:42.419255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1364/2731] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:42 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:52.415016 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=50 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:53.040678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:53.123676 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa60502-6bd3-44c2-abef-c7533c341e02 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:53.129508 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:53.130182 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.130462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.130743 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] PATH_INFO: `/v3/users/197918379fad428893add837f3024678` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.132021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=197918379fad428893add837f3024678)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:53.134189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:53.196209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '197918379fad428893add837f3024678'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.206007 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '197918379fad428893add837f3024678'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.222017 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '197918379fad428893add837f3024678'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.229495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '197918379fad428893add837f3024678'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.230349 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '197918379fad428893add837f3024678'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.240212 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1368/2732] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:37:53 2022] DELETE /identity/v3/users/197918379fad428893add837f3024678 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:53.245508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.245796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.246074 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.267691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:53.268983 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:53.335278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1365/2733] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:37:53.346404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:53.347848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:53.407197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268f01-f480-4bd9-8cc7-c8429d00d843 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:53.410877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:53.411505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.411787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.412090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.422878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:53.424395 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb5096-56a9-477e-ad72-6242f030d191 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:53.434503 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1369/2734] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:37:53.713866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:53.715317 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:53.728324 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29071-7fcc-4f8c-9fe5-857e734391ee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:53.733038 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:53.733700 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.734421 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.734582 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] PATH_INFO: `/v3/projects/a3928eed5c394bc099903b4bc6f7ed20` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.736509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3928eed5c394bc099903b4bc6f7ed20)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:53.738419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:53.791089 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3928eed5c394bc099903b4bc6f7ed20', 'request_id': 'req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.798991 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3928eed5c394bc099903b4bc6f7ed20', 'request_id': 'req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.808776 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3928eed5c394bc099903b4bc6f7ed20', 'request_id': 'req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.820613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] The token cache is being invalidate because project a3928eed5c394bc099903b4bc6f7ed20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:37:53.820898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.821541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-43b29303-ac92-4b7d-a7db-ee1a41f0e25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:37:53.823152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1366/2735] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:37:53 2022] DELETE /identity/v3/projects/a3928eed5c394bc099903b4bc6f7ed20 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:53.828296 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.828604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.828948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.853592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:53.854812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:53.875742 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:53.925623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1370/2736] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:37:53.933809 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.934100 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.934386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.955152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:53.956417 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:53.966154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1367/2737] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:53.971140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:53.981369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54660b4-1d78-41fd-9dad-14ee4b6eb9c1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:53.986480 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:53.987105 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:53.987390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:53.987803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:53.988552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:37:53.988856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:53.990467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.003076 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1371/2738] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:37:54.007770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.023018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1e848-7c42-42dc-ac78-1e1461e9f289 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.027755 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.028409 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.028687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.028971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.029640 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.032195 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.060606 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1368/2739] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:37:54 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:54.064685 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.075302 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096f27e-e110-4b0c-9174-0e1376e10953 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.134399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.135025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.135310 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.135588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.136319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.138278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.160863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1372/2740] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:37:54 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:37:54.164708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.174984 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352c6af-3957-4e9f-9571-5faff444189a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.179545 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.180181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.180462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.180998 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.181674 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.183249 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.193083 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1369/2741] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:54.196962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.207559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e20e6-6a97-4b42-aa77-6376f25bdd5e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.212259 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.212864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.213146 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.213424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.214115 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.215707 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee07da0-827c-4d4e-b09d-ed8c34c7c854 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.227776 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1373/2742] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:37:54.232847 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.243631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db238e55-cc07-4732-8668-1806e64ae672 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.248316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.248987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.249269 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.249569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] PATH_INFO: `/v3/projects/0fa80893306049cd8e4ca7e64ff1e5de/users/7bf8cd39d82348fa8501c50b29034cd0/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.264596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa80893306049cd8e4ca7e64ff1e5de, user_id=7bf8cd39d82348fa8501c50b29034cd0, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.267859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.282006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1370/2743] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:37:54 2022] PUT /identity/v3/projects/0fa80893306049cd8e4ca7e64ff1e5de/users/7bf8cd39d82348fa8501c50b29034cd0/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:37:54.296458 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.296748 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.297700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.327027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] MFA Rules not processed for user `7bf8cd39d82348fa8501c50b29034cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:37:54.328266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:54.396768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1374/2744] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:37:54.408154 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:54.409969 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:54.432486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:54.475396 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e1a38-73b8-40c1-84a7-db7d016b81c1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:54.479192 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:54.479815 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:54.480123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:54.480401 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:54.489276 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:54.490718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:54.499612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1371/2745] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:37:57.949774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:37:57.951344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:57.973074 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:37:58.021338 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c42912-ef8e-4c46-8e49-d472a9df135f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:37:58.025580 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:37:58.026316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:37:58.026653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:37:58.027214 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:37:58.029945 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:37:58.098858 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:37:58.100369 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb2d40-5509-424b-bebf-3d4f258cf97b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:37:58.110830 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1375/2746] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:38:04.495891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=17 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:05.137752 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:05.159482 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e236391-053c-407c-b20a-8c8fe9a29d3c service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:05.167343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:05.168347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:05.168780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:05.169190 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] PATH_INFO: `/v3/users/7bf8cd39d82348fa8501c50b29034cd0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:05.171794 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf8cd39d82348fa8501c50b29034cd0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:05.175736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:05.249971 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf8cd39d82348fa8501c50b29034cd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:05.250843 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf8cd39d82348fa8501c50b29034cd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:05.260704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf8cd39d82348fa8501c50b29034cd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:05.270528 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf8cd39d82348fa8501c50b29034cd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:05.285151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf8cd39d82348fa8501c50b29034cd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:05.299340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1372/2747] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:38:05 2022] DELETE /identity/v3/users/7bf8cd39d82348fa8501c50b29034cd0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:38:05.300785 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:05.301241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:05.301360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:05.346465 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:38:05.348761 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:05.432573 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1376/2748] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:38:05.458961 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:05.460550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:05.564087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8ccdf-7f94-4a35-b536-9706b371769a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:05.582541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:05.582541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:05.582541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:05.582541 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:05.595948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:05.597347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:05.631981 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1373/2749] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:38:05.903278 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:05.906808 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:05.932850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aeda6c-d01d-4733-97d4-93a76ac3ebd1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:05.940600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:05.941758 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:05.942367 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:05.942799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] PATH_INFO: `/v3/users/d737eac8ae0343df9ace919d75596800` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:05.944690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d737eac8ae0343df9ace919d75596800)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:05.947559 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:06.022017 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:06.023902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:06.029261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd737eac8ae0343df9ace919d75596800'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.042498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd737eac8ae0343df9ace919d75596800'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.051583 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2518-60d2-4322-91ec-e91e474f5365 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:06.052878 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:06.062666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd737eac8ae0343df9ace919d75596800'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.064728 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:06.065457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:06.065748 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:06.066024 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] PATH_INFO: `/v3/projects/0fa80893306049cd8e4ca7e64ff1e5de` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:06.067465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa80893306049cd8e4ca7e64ff1e5de)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:06.070561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:06.084699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd737eac8ae0343df9ace919d75596800'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.085983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd737eac8ae0343df9ace919d75596800'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.096255 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1377/2750] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:38:05 2022] DELETE /identity/v3/users/d737eac8ae0343df9ace919d75596800 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:38:06.148840 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa80893306049cd8e4ca7e64ff1e5de', 'request_id': 'req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.157812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa80893306049cd8e4ca7e64ff1e5de', 'request_id': 'req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.171393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa80893306049cd8e4ca7e64ff1e5de', 'request_id': 'req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.183773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:06.194002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] The token cache is being invalidate because project 0fa80893306049cd8e4ca7e64ff1e5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:38:06.194413 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.195677 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.204092 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1374/2751] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:38:06 2022] DELETE /identity/v3/projects/0fa80893306049cd8e4ca7e64ff1e5de => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:38:06.331311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:06.334170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:06.429159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f3094-1ad0-4227-9c17-30862af16ee9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:06.527734 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:06.528499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:06.528884 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:06.529225 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] PATH_INFO: `/v3/projects/d3fd186ce4ca4030b191309f43d43baa` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:06.531059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3fd186ce4ca4030b191309f43d43baa)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:06.533127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:06.582661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3fd186ce4ca4030b191309f43d43baa', 'request_id': 'req-f5620735-0e48-42c4-bb77-9a9c355c887b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.589090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3fd186ce4ca4030b191309f43d43baa', 'request_id': 'req-f5620735-0e48-42c4-bb77-9a9c355c887b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.601649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3fd186ce4ca4030b191309f43d43baa', 'request_id': 'req-f5620735-0e48-42c4-bb77-9a9c355c887b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.616913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] The token cache is being invalidate because project d3fd186ce4ca4030b191309f43d43baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:38:06.617301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.618140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.627399 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1378/2752] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:38:06 2022] DELETE /identity/v3/projects/d3fd186ce4ca4030b191309f43d43baa => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:38:06.628698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:06.630357 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:06.644420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ad6dd-a0a7-4f65-aa84-06d60ce2e1f6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:06.716345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:06.717239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:06.717730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:06.718758 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] PATH_INFO: `/v3/users/7ddfadcef85f49d6a4156ee1dccba56c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:06.720365 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddfadcef85f49d6a4156ee1dccba56c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:06.722433 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:06.788299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddfadcef85f49d6a4156ee1dccba56c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.788299 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddfadcef85f49d6a4156ee1dccba56c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.798948 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddfadcef85f49d6a4156ee1dccba56c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.819066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddfadcef85f49d6a4156ee1dccba56c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.839412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-808d8065-4506-41c5-8741-f7a33fea358e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddfadcef85f49d6a4156ee1dccba56c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:06.853172 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1375/2753] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:38:06 2022] DELETE /identity/v3/users/7ddfadcef85f49d6a4156ee1dccba56c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:38:07.060704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:07.063030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:07.077353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620735-0e48-42c4-bb77-9a9c355c887b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:07.092223 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:07.154027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:07.154711 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:07.154997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:07.155281 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] PATH_INFO: `/v3/projects/bf5c5a38ffe540ebada04f8311310cd8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:07.157561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5c5a38ffe540ebada04f8311310cd8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:07.160584 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:07.230704 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5c5a38ffe540ebada04f8311310cd8', 'request_id': 'req-921a8e47-eda3-4168-81a9-0f713f733526'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:07.238608 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5c5a38ffe540ebada04f8311310cd8', 'request_id': 'req-921a8e47-eda3-4168-81a9-0f713f733526'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:07.247547 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5c5a38ffe540ebada04f8311310cd8', 'request_id': 'req-921a8e47-eda3-4168-81a9-0f713f733526'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:07.260397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] The token cache is being invalidate because project bf5c5a38ffe540ebada04f8311310cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:38:07.260829 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:07.261485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:38:07.263012 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1379/2754] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:38:07 2022] DELETE /identity/v3/projects/bf5c5a38ffe540ebada04f8311310cd8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:38:08.692795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:08.693343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:08.693343 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:08.713829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] MFA Rules not processed for user `691d2cd293b244398bc9a3a98f1ad928`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:38:08.715033 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:08.772691 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1376/2755] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:38:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:38:17.257049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:18.765988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:38:58.576148 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:38:58.577669 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:58.640098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8e47-eda3-4168-81a9-0f713f733526 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:38:58.643674 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:38:58.644312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:38:58.644596 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:38:58.644866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:38:58.646558 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:38:58.730243 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:38:58.732683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:38:58.744221 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1380/2756] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:38:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:06.863088 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:06.864530 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:06.877865 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7682116-cb1f-4ff3-b076-4f37c1f7062a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:06.881438 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:06.882058 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:06.882334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:06.882604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:06.884374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:06.898767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:06.900021 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:06.917655 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1377/2757] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:08.739179 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=54 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:11.022311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:11.024047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:11.043388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10095b2-22c7-4c41-9186-55b071e48dd4 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:11.052141 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:11.052824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:11.053107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:11.053645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:11.055572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:11.071258 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:11.072983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:11.088317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1381/2758] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:16.906708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:19.465804 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:19.468060 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:19.545693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db458-bc95-4b02-815c-82e242fb57bd service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:19.550265 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:19.550911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:19.551189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:19.551465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] PATH_INFO: `/v3/users/691d2cd293b244398bc9a3a98f1ad928` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:19.552689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=691d2cd293b244398bc9a3a98f1ad928)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:19.554871 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:19.611673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691d2cd293b244398bc9a3a98f1ad928'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.612389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691d2cd293b244398bc9a3a98f1ad928'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.620668 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691d2cd293b244398bc9a3a98f1ad928'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.627799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691d2cd293b244398bc9a3a98f1ad928'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.641807 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c01791c-8131-494e-9a4a-35babefe3edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691d2cd293b244398bc9a3a98f1ad928'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.647754 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1378/2759] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:19 2022] DELETE /identity/v3/users/691d2cd293b244398bc9a3a98f1ad928 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:19.834885 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:19.847691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b352-8387-4ed9-82ef-a6392db75dbf service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:19.907854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:19.908519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:19.908805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:19.909083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] PATH_INFO: `/v3/projects/33ea04cddf084bc9b4aab1e7c477abbf` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:19.910249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ea04cddf084bc9b4aab1e7c477abbf)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:19.912194 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:19.959623 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ea04cddf084bc9b4aab1e7c477abbf', 'request_id': 'req-1d526d44-c875-4717-acc8-b1ef327c4554'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.965142 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ea04cddf084bc9b4aab1e7c477abbf', 'request_id': 'req-1d526d44-c875-4717-acc8-b1ef327c4554'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.973152 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ea04cddf084bc9b4aab1e7c477abbf', 'request_id': 'req-1d526d44-c875-4717-acc8-b1ef327c4554'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.984850 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] The token cache is being invalidate because project 33ea04cddf084bc9b4aab1e7c477abbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:19.985137 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.985781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1d526d44-c875-4717-acc8-b1ef327c4554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:19.987294 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1382/2760] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:19 2022] DELETE /identity/v3/projects/33ea04cddf084bc9b4aab1e7c477abbf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:19.992151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:19.992434 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:19.992706 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.014616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:20.015890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:20.075627 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1379/2761] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:39:20.085098 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.085400 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.085672 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.106266 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:20.108063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:20.115974 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1383/2762] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:20.120132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.130885 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34db8bc-64f6-478b-bf4c-c68bf2bf8e37 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.135578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.136230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.136513 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.136787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.137519 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:20.137860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.139480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.151450 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1380/2763] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:20.156066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.166164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541de67-951b-4af2-811c-5d397d758c59 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.171471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.172118 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.172398 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.172666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.173336 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.175247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.214383 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1384/2764] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:20.218642 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.234122 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ff24-fffb-47c5-9a6f-07168effb970 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.237730 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:20.292535 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.293168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.293452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.293727 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.294397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.296895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.319331 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1381/2765] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:20.323518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.334293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfa704-a8a0-4e68-90b9-c8a7110b8acc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.339038 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.339649 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.339964 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.340248 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.340948 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.342572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.353161 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1385/2766] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:20.361123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.371302 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452313-26f0-48c3-b38b-5dd2a1ef7386 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.376468 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.377081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.377363 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.377736 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.378359 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.379990 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e078-8037-4e68-85d5-883979755b40 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.391076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1382/2767] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:20.395556 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.407781 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd6f4-cc3f-4ba4-a198-68ac21e2c6b1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.413249 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.413956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.414195 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.414493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] PATH_INFO: `/v3/projects/feac57c781624582a04ff1e35173a21b/users/30f518f7c5924e688b7157a21f5e74c0/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.434047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feac57c781624582a04ff1e35173a21b, user_id=30f518f7c5924e688b7157a21f5e74c0, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.437397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.449609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1386/2768] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:20 2022] PUT /identity/v3/projects/feac57c781624582a04ff1e35173a21b/users/30f518f7c5924e688b7157a21f5e74c0/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:20.459943 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.460231 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.460507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.489689 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] MFA Rules not processed for user `30f518f7c5924e688b7157a21f5e74c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:20.490881 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:20.553768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1383/2769] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:20.558460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.564220 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:20.621058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d1048-eb4d-45df-8903-b5cc16f27122 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.625798 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.626366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.626642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.626923 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.627587 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.629532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.658688 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1387/2770] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:20.663327 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.675926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1282b9-ae97-463b-b7b4-235567d1ea22 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.730923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.731532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.731812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.732114 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.733394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.735278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.755567 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1384/2771] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:20 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:20.759881 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.770090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5a048-406a-425c-963b-b7683b10337f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.774657 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.775251 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.775530 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.775803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.776519 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.778107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.792617 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1388/2772] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:20.794134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.801638 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:20.807012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54080-8515-4e96-b323-534e24fc8b60 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.813308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.813999 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.814278 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.814550 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] PATH_INFO: `/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0/users/fd2575ffeed041f495859c84f41cb79a/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.831397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd731d3a58ad486e8a6bb17e3f542bd0, user_id=fd2575ffeed041f495859c84f41cb79a, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.834174 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.850344 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1385/2773] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:20 2022] PUT /identity/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0/users/fd2575ffeed041f495859c84f41cb79a/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:20.854460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.867726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d646-0156-46df-afad-c34260dfb642 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.944636 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.945593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.946061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.946597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.947931 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.951499 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa2fc9-5597-4604-a8ca-258a390e8f01 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:20.972319 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1389/2774] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:39:20.976400 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:20.987003 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5bb4-add2-4c14-90c4-ba1e5a9da410 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:20.991756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:20.992594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:20.992884 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:20.993153 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] PATH_INFO: `/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0/users/fd2575ffeed041f495859c84f41cb79a/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:20.995132 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd731d3a58ad486e8a6bb17e3f542bd0, user_id=fd2575ffeed041f495859c84f41cb79a, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:20.997879 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:21.009152 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1386/2775] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:20 2022] PUT /identity/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0/users/fd2575ffeed041f495859c84f41cb79a/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:21.020124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:21.020555 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:21.020960 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:21.053493 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] MFA Rules not processed for user `fd2575ffeed041f495859c84f41cb79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:21.054790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:21.124977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:21.129316 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1390/2776] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:21.142048 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:21.143779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:21.207318 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f1def-65b0-4a5d-b4cb-4d9228bbfc67 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:21.211030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:21.211657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:21.212500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:21.212500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:21.222950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:21.224483 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:21.234768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1387/2777] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:21.313058 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:21.323437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ae2e-724a-4e56-8d5f-fb2d6558dd8c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:21.327932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:21.328548 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:21.328824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:21.329092 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:21.517236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:21.518576 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5453d-f5c6-4761-87fe-aa568a363d6c service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:21.527692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1391/2778] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:24.255404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:24.319232 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53aa3-e9ee-48f4-8703-51bf28179671 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:24.323905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:24.324554 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:24.324841 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:24.325120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] PATH_INFO: `/v3/users/30f518f7c5924e688b7157a21f5e74c0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:24.326926 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f518f7c5924e688b7157a21f5e74c0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:24.328910 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:24.374987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f518f7c5924e688b7157a21f5e74c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.375650 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f518f7c5924e688b7157a21f5e74c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.383013 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f518f7c5924e688b7157a21f5e74c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.390667 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f518f7c5924e688b7157a21f5e74c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.397558 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:24.404059 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f518f7c5924e688b7157a21f5e74c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.411459 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1388/2779] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:24 2022] DELETE /identity/v3/users/30f518f7c5924e688b7157a21f5e74c0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:24.415993 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:24.416488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:24.416689 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:24.435797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:24.437033 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:24.496486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1392/2780] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:24.506586 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:24.508022 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:24.566091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91218fc7-fd00-4824-8b7c-f31f734f2dc7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:24.569970 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:24.570687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:24.570987 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:24.571253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:24.581053 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:24.582479 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:24.592467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1389/2781] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:24.863047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:24.864589 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:24.879485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c71f5f-d859-400f-8dc4-c2b81ac65eb5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:24.884090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:24.884726 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:24.885006 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:24.885275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] PATH_INFO: `/v3/projects/feac57c781624582a04ff1e35173a21b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:24.886413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] RBAC: Authorizing `identity:delete_project(project_id=feac57c781624582a04ff1e35173a21b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:24.890326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:24.936564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feac57c781624582a04ff1e35173a21b', 'request_id': 'req-87474a4d-5fcc-49cf-8ece-42148829488e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.942222 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feac57c781624582a04ff1e35173a21b', 'request_id': 'req-87474a4d-5fcc-49cf-8ece-42148829488e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.949959 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feac57c781624582a04ff1e35173a21b', 'request_id': 'req-87474a4d-5fcc-49cf-8ece-42148829488e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.961799 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] The token cache is being invalidate because project feac57c781624582a04ff1e35173a21b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:24.962114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.962750 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:24.964305 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1393/2782] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:24 2022] DELETE /identity/v3/projects/feac57c781624582a04ff1e35173a21b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:24.969206 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:24.978354 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac705c46-0327-4d7a-8f62-5a006126b966 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.038767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.038767 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.039167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.039167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] PATH_INFO: `/v3/users/fd2575ffeed041f495859c84f41cb79a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.040301 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2575ffeed041f495859c84f41cb79a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.042225 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.091594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2575ffeed041f495859c84f41cb79a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.092791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2575ffeed041f495859c84f41cb79a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.102294 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2575ffeed041f495859c84f41cb79a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.113227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2575ffeed041f495859c84f41cb79a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.126102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-98952305-897e-42bc-95ca-657e6f8b68ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2575ffeed041f495859c84f41cb79a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.132561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1390/2783] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:24 2022] DELETE /identity/v3/users/fd2575ffeed041f495859c84f41cb79a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:25.426523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:25.426523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:25.436572 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87474a4d-5fcc-49cf-8ece-42148829488e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.457229 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:25.498911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.499564 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.499870 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.500156 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] PATH_INFO: `/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.501330 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd731d3a58ad486e8a6bb17e3f542bd0)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.503285 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.550541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd731d3a58ad486e8a6bb17e3f542bd0', 'request_id': 'req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.556384 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd731d3a58ad486e8a6bb17e3f542bd0', 'request_id': 'req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.564839 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd731d3a58ad486e8a6bb17e3f542bd0', 'request_id': 'req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.577068 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] The token cache is being invalidate because project cd731d3a58ad486e8a6bb17e3f542bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:25.577352 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.577977 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e4031aad-f4ee-4aaf-a3f5-e812bf2c1983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:25.579462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1394/2784] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:25 2022] DELETE /identity/v3/projects/cd731d3a58ad486e8a6bb17e3f542bd0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:25.585283 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.585791 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.585898 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.605918 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:25.607131 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:25.612676 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:25.668660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1391/2785] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:25.732643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.732930 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.733210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.752703 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:25.753918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:25.761775 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1395/2786] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:25.766737 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:25.776599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126ed9d-e1cb-41bf-85c7-aa5200218808 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.781240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.781854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.782136 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.782412 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.783118 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:25.783546 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.785196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.797143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1392/2787] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:25.801570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:25.811847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f67bf-079c-4fda-afc5-45281fa99f21 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.816710 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.817332 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.817610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.817880 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.818552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.820479 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.850806 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1396/2788] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:25 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:25.855215 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:25.865165 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dac68-fed0-45df-877e-b57728aeefb8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.923571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.924219 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.924503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.924779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.925472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.927373 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.955106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1393/2789] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:25 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:25.959226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:25.969894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c41a0-ed22-4e84-8d9d-bf85f3b9b6eb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:25.974597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:25.975216 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:25.975498 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:25.975768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:25.976571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:25.978168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:25.988449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1397/2790] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:25.992563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.003255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1753-52ed-44b4-8d0e-52bcf1891156 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.007893 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.008503 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.008785 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.009061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.009749 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.011337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a869b3f-77e4-43c7-b6e7-8f34ec8a7b0e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.022712 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1394/2791] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:26.026926 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.037086 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c096d-efa2-4905-a1d2-740562665620 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.041691 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.042452 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.042752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.043022 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] PATH_INFO: `/v3/projects/e6d223ceae2c4380b7b07980e73e2ad2/users/517151d80c3a46a6b707c1b187610070/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.059557 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d223ceae2c4380b7b07980e73e2ad2, user_id=517151d80c3a46a6b707c1b187610070, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.062991 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.081366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1398/2792] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:26 2022] PUT /identity/v3/projects/e6d223ceae2c4380b7b07980e73e2ad2/users/517151d80c3a46a6b707c1b187610070/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:39:26.091282 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.091572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.091902 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.120684 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] MFA Rules not processed for user `517151d80c3a46a6b707c1b187610070`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:26.121911 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:26.188878 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1395/2793] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:26.193339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.241849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:26.258659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c39418-963f-43e5-8315-87bd4f2cc75e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.263345 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.264014 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.264297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.264571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.265246 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.267189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.296114 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1399/2794] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:26 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:26.300247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.312424 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cb410-9ee6-48d1-bad0-164b474682f1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.316067 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:26.370380 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.371006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.371292 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.371572 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.372291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.374844 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.396559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1396/2795] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:26 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:26.400877 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.412140 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb7cb-c13d-4157-8012-f78cfcdc46b8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.416949 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.417570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.417847 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.418126 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.418823 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.420983 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.431752 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1400/2796] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:26.436822 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.447271 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ce2dc-3bd7-491a-8203-1e53914e4adf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.452126 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.452803 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.453086 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.453364 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] PATH_INFO: `/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd/users/0091df738b364e4ab3eef7157734e856/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.469573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8a6b3ad7d242318b76617c6775ecdd, user_id=0091df738b364e4ab3eef7157734e856, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.472288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.486958 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1397/2797] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:26 2022] PUT /identity/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd/users/0091df738b364e4ab3eef7157734e856/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:26.491957 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.504697 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92466b4a-7f77-4d21-a5d4-0fdbd05dc139 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.561824 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.562507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.562718 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.562992 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.563812 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.565435 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef635c-0217-464c-b426-72c07931a489 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.575297 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1401/2798] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:39:26.579673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.590056 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913d7c-22b1-4cf5-94b2-bd5cc40ea902 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.595216 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.595917 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.596199 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.596471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] PATH_INFO: `/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd/users/0091df738b364e4ab3eef7157734e856/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.598394 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8a6b3ad7d242318b76617c6775ecdd, user_id=0091df738b364e4ab3eef7157734e856, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.601066 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.612511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1398/2799] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:26 2022] PUT /identity/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd/users/0091df738b364e4ab3eef7157734e856/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:39:26.623218 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.623512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.623851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.654797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] MFA Rules not processed for user `0091df738b364e4ab3eef7157734e856`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:26.656059 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:26.723659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1402/2800] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 15:39:26.739502 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:26.741334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:26.802252 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786e156-5798-4cf0-9c40-b9a5c2f2604a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:26.806239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:26.806856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:26.807133 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:26.807429 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:26.816783 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:26.818138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:26.828268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1399/2801] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:27.157379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.166976 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22656dc6-3b2c-4138-9c3e-138a6b9c54d2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:27.171894 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0091df738b364e4ab3eef7157734e856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e8a6b3ad7d242318b76617c6775ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:27.172497 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.172783 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.173054 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.173751 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:27.174053 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:27.175643 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:27.187391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1403/2802] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:27.191682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.201634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ec6e-7153-4c12-aa0b-bde4ae62cbe7 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:27.206360 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0091df738b364e4ab3eef7157734e856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e8a6b3ad7d242318b76617c6775ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:27.206944 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.207234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.207515 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.208224 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:27.210106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:27.237205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1400/2803] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:27 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:27.241520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.251804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878db95-269c-45a7-b80d-abe5dccffe74 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:27.280106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:27.308434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0091df738b364e4ab3eef7157734e856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e8a6b3ad7d242318b76617c6775ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:27.309044 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.309324 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.309598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.310284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:27.311123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:27.312863 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:27.324724 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1404/2804] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:27.360980 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.371076 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db63a-7de0-4208-987a-c50673311e2b tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:27.376253 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0091df738b364e4ab3eef7157734e856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e8a6b3ad7d242318b76617c6775ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:27.376886 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.377171 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.377473 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] PATH_INFO: `/v3/projects/6321e60c23d847ebb0b87f705dad8e09` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.378632 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6321e60c23d847ebb0b87f705dad8e09)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:27.380577 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:27.418306 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:27.430508 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6321e60c23d847ebb0b87f705dad8e09', 'request_id': 'req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.439268 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6321e60c23d847ebb0b87f705dad8e09', 'request_id': 'req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.448263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6321e60c23d847ebb0b87f705dad8e09', 'request_id': 'req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.459524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] The token cache is being invalidate because project 6321e60c23d847ebb0b87f705dad8e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:27.459812 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.460518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b4a7c962-7d22-4c71-81d6-efa7b48e31c8 tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.463054 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1401/2805] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:27 2022] DELETE /identity/v3/projects/6321e60c23d847ebb0b87f705dad8e09 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:39:27.686732 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.688496 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:27.753972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c5829-3958-4cbd-ade4-656720a56b1a tempest-VolumeQuotaClassesTest-756756951 tempest-VolumeQuotaClassesTest-756756951-project-admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:27.809386 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:27.810040 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.810416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.811360 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] PATH_INFO: `/v3/users/517151d80c3a46a6b707c1b187610070` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.812616 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517151d80c3a46a6b707c1b187610070)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:27.814552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:27.860327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517151d80c3a46a6b707c1b187610070'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.867795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517151d80c3a46a6b707c1b187610070'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.881334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517151d80c3a46a6b707c1b187610070'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.883350 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:27.889027 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517151d80c3a46a6b707c1b187610070'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.890003 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517151d80c3a46a6b707c1b187610070'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:27.896779 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1405/2806] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:27 2022] DELETE /identity/v3/users/517151d80c3a46a6b707c1b187610070 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:27.901263 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:27.901756 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:27.901863 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:27.921203 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:27.922714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:27.986196 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1402/2807] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:27.996091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:27.997520 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:28.054980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2388d-c883-4d03-ae90-26197b61e603 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:28.058803 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:28.059418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:28.059723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:28.060011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:28.069540 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:28.070994 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:28.080719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1406/2808] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:28.356620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:28.358109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:28.370345 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e70f-948c-4850-aa8b-ac7fcf15cf92 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:28.375202 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:28.376108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:28.376387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:28.376661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] PATH_INFO: `/v3/projects/e6d223ceae2c4380b7b07980e73e2ad2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:28.377837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d223ceae2c4380b7b07980e73e2ad2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:28.379768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:28.428613 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d223ceae2c4380b7b07980e73e2ad2', 'request_id': 'req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.434860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d223ceae2c4380b7b07980e73e2ad2', 'request_id': 'req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.442771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d223ceae2c4380b7b07980e73e2ad2', 'request_id': 'req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.453763 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] The token cache is being invalidate because project e6d223ceae2c4380b7b07980e73e2ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:28.454050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.454701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.456242 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1403/2809] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:28 2022] DELETE /identity/v3/projects/e6d223ceae2c4380b7b07980e73e2ad2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:28.460226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:28.470552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54508f66-260e-46e6-8c3c-e328fecd7097 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:28.527390 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:28.528063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:28.528343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:28.528620 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] PATH_INFO: `/v3/users/0091df738b364e4ab3eef7157734e856` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:28.529777 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0091df738b364e4ab3eef7157734e856)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:28.532387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:28.577876 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0091df738b364e4ab3eef7157734e856'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.585107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0091df738b364e4ab3eef7157734e856'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.597364 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0091df738b364e4ab3eef7157734e856'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.604087 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0091df738b364e4ab3eef7157734e856'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.604755 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d3b94607-1a36-45d9-b204-6d5e331239f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0091df738b364e4ab3eef7157734e856'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:28.610575 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1407/2810] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:28 2022] DELETE /identity/v3/users/0091df738b364e4ab3eef7157734e856 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:28.899596 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:28.901145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:28.916123 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52bb1-333a-47b7-a7f5-0c49f3bf8874 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:28.967240 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:28.982564 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:28.983214 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:28.983493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:28.983789 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] PATH_INFO: `/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:28.985044 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8a6b3ad7d242318b76617c6775ecdd)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:28.986940 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.033916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8a6b3ad7d242318b76617c6775ecdd', 'request_id': 'req-b196128b-1f64-4ed1-9148-fa754bbc09c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:29.040230 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8a6b3ad7d242318b76617c6775ecdd', 'request_id': 'req-b196128b-1f64-4ed1-9148-fa754bbc09c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:29.048659 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8a6b3ad7d242318b76617c6775ecdd', 'request_id': 'req-b196128b-1f64-4ed1-9148-fa754bbc09c8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:29.059795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] The token cache is being invalidate because project 8e8a6b3ad7d242318b76617c6775ecdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:29.060106 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:29.060753 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b196128b-1f64-4ed1-9148-fa754bbc09c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:29.062882 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1404/2811] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:28 2022] DELETE /identity/v3/projects/8e8a6b3ad7d242318b76617c6775ecdd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:29.067895 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.068424 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.068523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.090595 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:29.091792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:29.100230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:29.156709 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1408/2812] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:29.164780 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.165069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.165348 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.186414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:29.188185 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:29.197571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1405/2813] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:29.201899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.211973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7b84d-090c-4040-8db6-83a47397378f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.216652 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.217188 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.217471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.217912 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.218647 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:29.218952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.220605 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.232456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1409/2814] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:29.237569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.246901 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c03c3-e2b4-46bb-a1ff-be2e73f21e75 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.252233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.252838 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.253120 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.253393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.254063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.255972 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.283200 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1406/2815] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:29.287457 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.298288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b55d1-0df8-43ba-9b58-675c8b433b7d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.354429 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.355062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.355361 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.355630 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.356329 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.358805 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.380759 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1410/2816] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:29.384941 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.395397 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c598-86f0-4409-a024-88feac40f6cc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.400012 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.400634 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.400914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.401187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.401877 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.403507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.414277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1407/2817] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:29.418706 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.429645 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f4f1a-a8d5-4a4b-b9bc-e33d52e3e14a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.434237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.434871 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.435150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.435422 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.436170 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.438366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76793b01-7da2-46c7-bb03-24c28decf6f0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.449831 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1411/2818] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:29.454186 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.464371 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfd97d-fc7a-4c69-aea3-2a248e0ad5bb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.469817 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.470422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.470715 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.470986 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] PATH_INFO: `/v3/projects/ccde8d1192124f4fbfd50bd73557be3f/users/518d5631cffe44b68af546ac782cb0e4/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.487631 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccde8d1192124f4fbfd50bd73557be3f, user_id=518d5631cffe44b68af546ac782cb0e4, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.490415 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.504255 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1408/2819] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:29 2022] PUT /identity/v3/projects/ccde8d1192124f4fbfd50bd73557be3f/users/518d5631cffe44b68af546ac782cb0e4/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:29.514700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.515035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.515312 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.543150 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] MFA Rules not processed for user `518d5631cffe44b68af546ac782cb0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:29.544397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:29.607677 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1412/2820] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:29.612787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.673234 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a444-4950-48b6-889c-8255181ce53e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.678043 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.678647 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.678930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.679205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.679891 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.681774 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.701233 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:29.711864 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1409/2821] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:29.713028 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.724339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828af00-8258-4090-842a-4f7f95914b06 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.728861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:29.781910 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.782541 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.782804 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.783075 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.783741 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.785860 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.808344 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1413/2822] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:29 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:29.812745 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.823471 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6987-28bb-47a3-b51d-c0d11740023b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.828685 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.829305 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.829590 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.829866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.830569 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.832212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.844151 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1410/2823] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:39:29.848619 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.859370 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e0d4-68c1-4cda-a8ec-891937cad1ec admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.864035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.864762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.865039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.865322 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] PATH_INFO: `/v3/projects/e29082923cd9485c8a8f35caee7e53ee/users/3fe4c15d95f14c43b940a35167d6e359/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.882143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29082923cd9485c8a8f35caee7e53ee, user_id=3fe4c15d95f14c43b940a35167d6e359, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.884846 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.901393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1414/2824] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:29 2022] PUT /identity/v3/projects/e29082923cd9485c8a8f35caee7e53ee/users/3fe4c15d95f14c43b940a35167d6e359/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:29.906099 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:29.916861 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d580d7-e67c-40d2-933b-5bfb1fea9e0f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:29.976046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:29.976594 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:29.976874 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:29.977149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:29.977872 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:29.979588 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623688-1e7c-4869-953d-7ae0bd47de6f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:29.989573 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1411/2825] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:29.993822 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:30.004425 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc706008-5011-40af-a05a-65543ba26fd5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:30.009071 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:30.009820 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:30.010102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:30.010377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] PATH_INFO: `/v3/projects/e29082923cd9485c8a8f35caee7e53ee/users/3fe4c15d95f14c43b940a35167d6e359/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:30.012413 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29082923cd9485c8a8f35caee7e53ee, user_id=3fe4c15d95f14c43b940a35167d6e359, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:30.015567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:30.027656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1415/2826] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:29 2022] PUT /identity/v3/projects/e29082923cd9485c8a8f35caee7e53ee/users/3fe4c15d95f14c43b940a35167d6e359/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:30.039920 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:30.040211 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:30.040489 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:30.070587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] MFA Rules not processed for user `3fe4c15d95f14c43b940a35167d6e359`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:30.071866 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:30.144818 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1412/2827] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:30.156317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:30.158107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:30.219477 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e81a8-d9c6-4503-9902-5a296383e3d6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:30.223061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:30.223702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:30.224020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:30.224293 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:30.281228 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:30.282571 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8945c75-6d77-4b8e-a01e-1401373de801 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:30.291817 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1416/2828] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:32.844057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:32.912287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c843d4-fb5f-4a27-837f-430baa279410 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:32.917356 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:32.917563 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:32.917786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:32.918055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] PATH_INFO: `/v3/users/518d5631cffe44b68af546ac782cb0e4` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:32.919235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=518d5631cffe44b68af546ac782cb0e4)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:32.921235 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:32.933257 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:32.974859 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518d5631cffe44b68af546ac782cb0e4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:32.975523 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518d5631cffe44b68af546ac782cb0e4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:32.983451 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518d5631cffe44b68af546ac782cb0e4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:32.990942 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518d5631cffe44b68af546ac782cb0e4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.003571 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518d5631cffe44b68af546ac782cb0e4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.017956 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1413/2829] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:32 2022] DELETE /identity/v3/users/518d5631cffe44b68af546ac782cb0e4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:33.022827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:33.023374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:33.023374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:33.042731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:33.043986 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:33.053866 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:33.104873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1417/2830] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:33.114821 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:33.116334 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:33.176498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fc5c8-29b3-4ffc-9a45-c827130e5a7e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:33.180771 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:33.181422 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:33.181831 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:33.183439 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:33.196138 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:33.197597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:33.211244 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1414/2831] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:33.484588 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:33.486062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:33.498355 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee769c05-785b-40f0-9595-72734ef1cf04 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:33.503746 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:33.504448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:33.504729 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:33.505000 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] PATH_INFO: `/v3/projects/ccde8d1192124f4fbfd50bd73557be3f` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:33.506184 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccde8d1192124f4fbfd50bd73557be3f)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:33.508135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:33.557845 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccde8d1192124f4fbfd50bd73557be3f', 'request_id': 'req-6082e9c2-67dc-4b99-842d-ddf781e00732'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.564005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccde8d1192124f4fbfd50bd73557be3f', 'request_id': 'req-6082e9c2-67dc-4b99-842d-ddf781e00732'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.572339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccde8d1192124f4fbfd50bd73557be3f', 'request_id': 'req-6082e9c2-67dc-4b99-842d-ddf781e00732'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.583848 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] The token cache is being invalidate because project ccde8d1192124f4fbfd50bd73557be3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:33.584160 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.584794 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.586311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1418/2832] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:33 2022] DELETE /identity/v3/projects/ccde8d1192124f4fbfd50bd73557be3f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:33.590455 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:33.600746 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17a18a-23ae-4294-bf0b-66be1d8075bc service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:33.663229 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:33.663786 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:33.664102 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:33.664379 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] PATH_INFO: `/v3/users/3fe4c15d95f14c43b940a35167d6e359` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:33.665589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe4c15d95f14c43b940a35167d6e359)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:33.667511 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:33.702829 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:33.717399 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe4c15d95f14c43b940a35167d6e359'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.718063 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe4c15d95f14c43b940a35167d6e359'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.725837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe4c15d95f14c43b940a35167d6e359'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.733404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe4c15d95f14c43b940a35167d6e359'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.745603 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0e3daacd-10a3-4b25-b5d5-ac1a63749371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe4c15d95f14c43b940a35167d6e359'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:33.752387 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1415/2833] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:33 2022] DELETE /identity/v3/users/3fe4c15d95f14c43b940a35167d6e359 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:34.038954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.040524 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:34.056328 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e9c2-67dc-4b99-842d-ddf781e00732 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.114609 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.115265 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.115561 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.115995 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] PATH_INFO: `/v3/projects/e29082923cd9485c8a8f35caee7e53ee` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.117020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29082923cd9485c8a8f35caee7e53ee)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.118924 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.159456 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:34.166264 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29082923cd9485c8a8f35caee7e53ee', 'request_id': 'req-42845e09-f694-49fa-9a25-3c9a2c8c61f9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:34.173066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29082923cd9485c8a8f35caee7e53ee', 'request_id': 'req-42845e09-f694-49fa-9a25-3c9a2c8c61f9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:34.180604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29082923cd9485c8a8f35caee7e53ee', 'request_id': 'req-42845e09-f694-49fa-9a25-3c9a2c8c61f9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:34.192317 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] The token cache is being invalidate because project e29082923cd9485c8a8f35caee7e53ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:34.192604 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:34.193250 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42845e09-f694-49fa-9a25-3c9a2c8c61f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:34.194790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1419/2834] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:34 2022] DELETE /identity/v3/projects/e29082923cd9485c8a8f35caee7e53ee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:34.199895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.200414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.200597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.220988 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:34.222205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:34.281352 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1416/2835] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:34.289318 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.289601 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.289873 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.311635 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:34.312980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:34.321326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1420/2836] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:34.325660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.336272 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5700e-5a4e-4848-a0b1-08f875dc0098 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.341008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.341624 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.341905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.342291 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.343616 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:34.343968 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.345576 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.357708 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1417/2837] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:34.362102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.371462 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bf4d7-5c45-4776-bf61-93ab9bdc5f3b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.376091 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.376708 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.376984 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.377257 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.377911 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.379802 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.408475 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1421/2838] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:34 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:34.412725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.423316 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149c39a-a7df-4584-b830-4993182a6829 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.481905 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.482521 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.482799 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.483669 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.484381 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.486290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.508161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1418/2839] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:34 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:34.512377 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.523861 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf40353-ff2d-45d7-87ad-bcb581a66f40 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.528450 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.529067 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.529343 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.529610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.530947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.532582 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.542818 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1422/2840] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:34.547141 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.557518 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304189a-c5c4-4f7a-9ff4-14e6af6daef2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.562351 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.563004 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.563287 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.563561 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.564300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.565890 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757bba2-4fa2-4910-b25d-1875a91bbce6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.578055 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1419/2841] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:34.583529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.594353 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891677a3-ed41-49f2-8615-8126dceb742f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.599089 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.599762 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.600072 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.600347 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] PATH_INFO: `/v3/projects/cadc46b48c824bfba28a4ecfb434e701/users/0269589b89e6471183b5a73dc5eebae9/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.615585 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cadc46b48c824bfba28a4ecfb434e701, user_id=0269589b89e6471183b5a73dc5eebae9, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.618354 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.632123 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1423/2842] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:34 2022] PUT /identity/v3/projects/cadc46b48c824bfba28a4ecfb434e701/users/0269589b89e6471183b5a73dc5eebae9/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:34.643010 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb0b-bd9c-47ef-9165-30b1c9d3d874 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.643300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb0b-bd9c-47ef-9165-30b1c9d3d874 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.643578 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83fb0b-bd9c-47ef-9165-30b1c9d3d874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.674609 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dd83fb0b-bd9c-47ef-9165-30b1c9d3d874 None None] MFA Rules not processed for user `0269589b89e6471183b5a73dc5eebae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:34.675773 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dd83fb0b-bd9c-47ef-9165-30b1c9d3d874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:34.680162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:34.743520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1420/2843] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:34.754952 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:34.756692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:34.816406 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc453866-c5dd-4124-b0df-1839ce5291d2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:34.820037 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '711efe02d15146dfb6a1a737abcee3a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:34.820654 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:34.820956 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:34.821231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:34.830713 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:34.832197 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:34.841567 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1424/2844] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:36.223168 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:36.223714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:36.223714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:36.243134 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] MFA Rules not processed for user `0269589b89e6471183b5a73dc5eebae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:36.244392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:36.252227 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1421/2845] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:36.984066 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:36.985700 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:37.061023 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16b6e-2ecb-4c4c-b0f2-fac0fc547bcc service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:37.068083 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:37.068942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:37.069244 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:37.069701 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:37.079552 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:37.092727 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:37.094388 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:37.099180 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:37.106467 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1425/2846] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:39.086695 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:39.089027 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:39.173277 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3fbcb-e3fc-447c-b935-ef4f5af5ed77 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:39.177061 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:39.177950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:39.178239 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:39.178500 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:39.180374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:39.191493 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:39.192963 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42fd6e-14a8-4e58-9ec8-3ca23ec108cc service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:39.204007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1422/2847] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 15:39:45.312679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:45.314610 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:45.411801 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc3e8-4f95-4ff9-bd62-8cc95154bd24 service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:45.417526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:45.418164 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:45.418449 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:45.418728 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:45.420570 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:45.434061 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:45.436267 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:45.447618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1426/2848] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:49.197724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=54 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:51.489585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7734e8-721d-4233-bfd0-05ec2f0c9515 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:51.490213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7734e8-721d-4233-bfd0-05ec2f0c9515 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:51.490213 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7734e8-721d-4233-bfd0-05ec2f0c9515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:51.551404 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9e7734e8-721d-4233-bfd0-05ec2f0c9515 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:51.552930 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9e7734e8-721d-4233-bfd0-05ec2f0c9515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:51.620002 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1423/2849] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:51.666514 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:51.667899 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:51.757577 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0c23-ccf9-4220-90cb-e9da90a92971 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:51.765523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:51.766550 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:51.766865 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:51.767147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:51.775922 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:51.777521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:51.786486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1427/2850] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:39:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:39:52.339420 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb16ee-e517-4763-9776-6db14a018774 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:52.339772 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb16ee-e517-4763-9776-6db14a018774 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:52.340290 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb16ee-e517-4763-9776-6db14a018774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:52.362414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5dfb16ee-e517-4763-9776-6db14a018774 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:52.363633 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5dfb16ee-e517-4763-9776-6db14a018774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:52.370912 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1424/2851] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:52.383628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:52.394397 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a798f-afbc-4f33-b368-95d33e702a7f service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:52.401298 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:52.401933 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:52.402209 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:52.402482 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:52.411702 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:52.413379 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:52.422404 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1428/2852] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:39:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:39:52.642932 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ecb3-777c-4fc3-b6a8-94621ec3e21c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:52.643467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ecb3-777c-4fc3-b6a8-94621ec3e21c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:52.643467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ecb3-777c-4fc3-b6a8-94621ec3e21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:52.663601 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-61f5ecb3-777c-4fc3-b6a8-94621ec3e21c None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:52.664860 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-61f5ecb3-777c-4fc3-b6a8-94621ec3e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:52.672612 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1425/2853] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:52.684740 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:52.697505 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e416539-ce7b-4aab-b9f2-48a3d0ef80b6 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:52.702518 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:52.703127 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:52.703416 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:52.703687 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:52.712292 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:52.713889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:52.723032 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1429/2854] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:39:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:39:53.049889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:53.050175 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:53.050449 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:53.070430 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:53.071645 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:53.079329 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1426/2855] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:39:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:53.092522 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:53.104618 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd9e1-2781-421d-992e-54be7c08fbd3 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:53.109970 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:53.110592 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:53.111011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:53.111288 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:53.123011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:53.124768 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c279d-dd78-4ae4-8249-d5686ab9ee2d service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:53.133790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1430/2856] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:39:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:39:55.266121 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:55.332921 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6045a16-d941-4fcf-a322-08a376c1bfda None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:55.337465 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:55.338155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:55.338425 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:55.338698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] PATH_INFO: `/v3/users/0269589b89e6471183b5a73dc5eebae9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:55.340091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0269589b89e6471183b5a73dc5eebae9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:55.342704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:55.389390 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:55.391057 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0269589b89e6471183b5a73dc5eebae9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.391887 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0269589b89e6471183b5a73dc5eebae9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.400161 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0269589b89e6471183b5a73dc5eebae9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.409698 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0269589b89e6471183b5a73dc5eebae9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.430770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0269589b89e6471183b5a73dc5eebae9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.436889 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1427/2857] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:39:55 2022] DELETE /identity/v3/users/0269589b89e6471183b5a73dc5eebae9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:55.441392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:55.441896 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:55.442025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:55.462915 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:55.464192 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:55.474124 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:55.527913 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1431/2858] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:55.538460 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:55.540285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:55.616049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cec54-0617-4ae7-816d-02ad2d3ac7a9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:55.616049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:55.616049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:55.616049 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:55.616452 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:55.624484 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:55.625996 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e793e69-f1ef-4fe2-9552-73ff4c497771 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:55.636105 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1428/2859] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:39:55.814800 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:55.816247 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:55.829932 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a819a5-3327-4663-948e-cfc543bc2979 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:55.834658 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:55.835523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:55.835908 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:55.836261 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] PATH_INFO: `/v3/projects/cadc46b48c824bfba28a4ecfb434e701` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:55.837525 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cadc46b48c824bfba28a4ecfb434e701)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:55.839512 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:55.890236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cadc46b48c824bfba28a4ecfb434e701', 'request_id': 'req-e96ce0e1-94ee-495e-b4c0-db4e72001569'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.896159 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cadc46b48c824bfba28a4ecfb434e701', 'request_id': 'req-e96ce0e1-94ee-495e-b4c0-db4e72001569'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.903997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cadc46b48c824bfba28a4ecfb434e701', 'request_id': 'req-e96ce0e1-94ee-495e-b4c0-db4e72001569'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.915433 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] The token cache is being invalidate because project cadc46b48c824bfba28a4ecfb434e701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:39:55.915719 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.916402 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e96ce0e1-94ee-495e-b4c0-db4e72001569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:39:55.917906 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1432/2860] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:39:55 2022] DELETE /identity/v3/projects/cadc46b48c824bfba28a4ecfb434e701 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:39:55.922892 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:55.923208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:55.923486 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:55.944108 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:55.945296 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:56.008036 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1429/2861] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:56.016205 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.016495 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.016774 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.036476 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:56.037722 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:56.046374 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1433/2862] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:56.050687 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.061637 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928149a3-d3e2-4cd6-a4ec-70ca0080651d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.066220 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.066837 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.067116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.067395 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.069030 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 15:39:56.069414 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.071285 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.084974 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1430/2863] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 15:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 15:39:56.089656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.099339 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03398c-bb8d-4f7e-8875-9a738116dcd8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.104094 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.104723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.105002 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.105275 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.105954 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.108393 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.139375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1434/2864] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:39:56 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:39:56.143704 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.154006 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199e77-28df-41f6-9dab-f825435665d1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.204098 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:56.214506 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.215135 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.215587 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.215952 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.216591 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.218714 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.241137 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1431/2865] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:39:56 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:39:56.245507 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.258679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e643bd-1004-4f9e-977c-e298f24ec22a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.263301 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.263950 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.264236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.264756 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.265453 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.267062 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.276955 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1435/2866] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 15:39:56.281090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.291064 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c7d2-af33-41d4-8815-f1e3d125b177 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.296173 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.296779 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.297062 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.297342 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.298039 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.299629 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d26f-45a4-4246-8f64-6e8bb66655dd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.311046 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1432/2867] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:39:56.315163 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.325099 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597c624-f738-438d-bee6-9d9f6240a782 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.329673 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.330320 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.330598 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.330869 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] PATH_INFO: `/v3/projects/f40879b7ec11451a82501bf14592c04a/users/658f2b944c994780b615b5c0affd4600/roles/63ebc483a8af4b2584cc1a6f03e1625e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.346049 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40879b7ec11451a82501bf14592c04a, user_id=658f2b944c994780b615b5c0affd4600, role_id=63ebc483a8af4b2584cc1a6f03e1625e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.348736 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.363485 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1436/2868] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:39:56 2022] PUT /identity/v3/projects/f40879b7ec11451a82501bf14592c04a/users/658f2b944c994780b615b5c0affd4600/roles/63ebc483a8af4b2584cc1a6f03e1625e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 15:39:56.368496 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f256-9796-45cc-aeba-c07e3ee1d340 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.368787 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f256-9796-45cc-aeba-c07e3ee1d340 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.369091 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f256-9796-45cc-aeba-c07e3ee1d340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.388478 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1518f256-9796-45cc-aeba-c07e3ee1d340 None None] MFA Rules not processed for user `feddce10ac514a48a58f8b731d31a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:39:56.389682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1518f256-9796-45cc-aeba-c07e3ee1d340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:56.448018 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1433/2869] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:39:56.457864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:39:56.459631 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:39:56.496025 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:39:56.519366 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752cd0-4511-4597-beae-ad75c6c85d66 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:39:56.523171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:39:56.523790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:39:56.524107 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:39:56.524382 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:39:56.534640 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:39:56.536111 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:39:56.546682 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1437/2870] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:00.839393 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:00.840191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:00.840191 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:00.876551 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] MFA Rules not processed for user `658f2b944c994780b615b5c0affd4600`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:40:00.877718 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:01.003300 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1434/2871] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:40:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:40:01.004656 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.016291 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dae283-268c-490b-83c6-530cf40de267 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.021157 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.021787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.022063 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.022334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.023045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.025030 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.055432 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1438/2872] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 15:40:01 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 15:40:01.059894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.077531 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da68b8b-7439-4835-b007-e3e643b434f2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.147557 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.148245 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.148532 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.148810 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.149537 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.151903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.161643 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:01.178834 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1435/2873] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 15:40:01 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 15:40:01.185731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.196045 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ee7ea-d99a-4d0a-9486-12e1062c0301 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.201980 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.202666 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.202946 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.203539 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.204284 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.205918 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.221460 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1439/2874] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:40:01.225495 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.237228 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43fbf1-a2d3-43b0-ab5e-157446b90ba2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.242384 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.243081 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.243367 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.243653 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] PATH_INFO: `/v3/projects/1b6573e415c34be2a6e743bac33ccdf2/users/dff09ec31d78433387b2a2d831b890f3/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.264419 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6573e415c34be2a6e743bac33ccdf2, user_id=dff09ec31d78433387b2a2d831b890f3, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.267383 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.285204 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1436/2875] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:40:01 2022] PUT /identity/v3/projects/1b6573e415c34be2a6e743bac33ccdf2/users/dff09ec31d78433387b2a2d831b890f3/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:40:01.289594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.303237 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e140d6-69b9-45b0-ba98-dcadcee13645 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.381939 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.382590 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.382864 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.383135 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.383851 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.385543 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.397334 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1440/2876] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 15:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 15:40:01.401725 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:01.412187 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851841-5b84-49f1-a52a-6461af789076 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:01.416790 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:01.417457 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:01.417735 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:01.418007 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] PATH_INFO: `/v3/projects/1b6573e415c34be2a6e743bac33ccdf2/users/dff09ec31d78433387b2a2d831b890f3/roles/5ea86f82718f4404bfc74b1781b9f673` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:01.420597 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6573e415c34be2a6e743bac33ccdf2, user_id=dff09ec31d78433387b2a2d831b890f3, role_id=5ea86f82718f4404bfc74b1781b9f673)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:01.423246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:01.434935 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1437/2877] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 15:40:01 2022] PUT /identity/v3/projects/1b6573e415c34be2a6e743bac33ccdf2/users/dff09ec31d78433387b2a2d831b890f3/roles/5ea86f82718f4404bfc74b1781b9f673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:40:03.454694 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:03.566359 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed87c0-0e75-4194-aa11-ff0e8eb47348 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:03.646300 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:03.647231 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:03.647515 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:03.647791 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:03.649699 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:03.662210 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:03.664090 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ebb73-f566-4391-ab4b-6d6fa91c751b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:03.668230 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:03.680942 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1441/2878] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:40:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:03.802145 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:03.820031 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84c9b0-960a-4f21-8629-a6c08b862c80 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:03.832529 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:03.833184 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:03.833467 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:03.833742 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:03.835664 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:03.981690 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:03.983026 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:04.009589 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1438/2879] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:40:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:07.596226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfffc-933b-4e84-8b44-6f3fcc58425e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:07.596735 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfffc-933b-4e84-8b44-6f3fcc58425e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:07.596859 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfffc-933b-4e84-8b44-6f3fcc58425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:07.631143 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c91cfffc-933b-4e84-8b44-6f3fcc58425e None None] MFA Rules not processed for user `dff09ec31d78433387b2a2d831b890f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:40:07.632738 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c91cfffc-933b-4e84-8b44-6f3fcc58425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:07.718198 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1442/2880] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:40:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:40:07.730781 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:07.732337 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:07.826846 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61237-57bf-4c0e-9014-5e7d5bae4ebb service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:07.830498 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f96ae3547b4be6bd6bb31c14903926', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:07.831111 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:07.831392 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:07.831661 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:07.868079 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:07.896839 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:07.898273 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:07.907657 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1439/2881] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:40:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:08.042327 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13351-9594-412c-b7a5-7020a1ae7769 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:08.042615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13351-9594-412c-b7a5-7020a1ae7769 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:08.042893 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13351-9594-412c-b7a5-7020a1ae7769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:08.065526 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-20d13351-9594-412c-b7a5-7020a1ae7769 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:40:08.066767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-20d13351-9594-412c-b7a5-7020a1ae7769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:08.127962 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1443/2882] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:40:08.140368 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:08.141875 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:08.218259 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87f4ac-fb1e-4f82-a797-65c073ab5972 service glance] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:08.223733 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:08.224389 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:08.224673 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:08.224950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:08.234212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:08.235793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd5d8b-5ad6-473c-8638-d979f713947e service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:08.244109 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1440/2883] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:40:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:40:16.957448 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:16.957997 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:16.958168 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:16.977325 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] MFA Rules not processed for user `658f2b944c994780b615b5c0affd4600`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:40:16.978554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:16.985723 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1444/2884] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 15:40:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:40:18.240795 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=44 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:19.686768 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:19.687472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:19.687472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:19.708604 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:40:19.709830 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:19.720072 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1441/2885] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 15:40:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:40:19.734615 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:19.751597 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c341e-5ef9-4cc8-a442-5747765eaa3d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:19.760827 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:19.761790 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:19.762039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:19.762410 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:19.775226 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:19.777554 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:19.790889 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1445/2886] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:40:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:40:29.702340 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:29.784005 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=69 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:44.586480 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:44.596208 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb3663-a3ec-46c7-84d7-05b6d429f60e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:44.599797 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:44.600463 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:44.600743 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:44.601020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:44.610427 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:44.611854 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:44.622116 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1442/2887] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:40:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:46.316936 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:40:46.318627 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:46.337169 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223340f-3655-48fc-9d57-6c96284d8d5d service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:40:46.343536 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe58a99358bc42aebeb126e5ef2cf21a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:40:46.344241 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:40:46.344521 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:40:46.344795 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:40:46.349375 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:40:46.365047 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:40:46.366690 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:40:46.379326 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1446/2888] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:40:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:40:54.617205 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:40:56.373553 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:41:47.078682 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:47.081870 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:47.201825 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063ede9-05f1-47a5-82ce-9cc2730a13c4 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:47.207246 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:47.207904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:47.208189 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:47.208462 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:47.210485 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:47.222507 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:47.224162 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:47.234923 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1443/2889] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:41:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:41:50.526162 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:50.527626 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:50.601011 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def88-2317-4771-aa9c-4e6fb003f7ef service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:50.605659 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:50.606311 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:50.606593 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:50.606892 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] PATH_INFO: `/v3/users/37a42563c3d34a689c3cec947d649fc1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:50.608080 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a42563c3d34a689c3cec947d649fc1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:50.610021 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:50.657189 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a42563c3d34a689c3cec947d649fc1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:50.664628 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a42563c3d34a689c3cec947d649fc1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:50.676797 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a42563c3d34a689c3cec947d649fc1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:50.683528 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a42563c3d34a689c3cec947d649fc1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:50.684235 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a42563c3d34a689c3cec947d649fc1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:50.690282 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1447/2890] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:41:50 2022] DELETE /identity/v3/users/37a42563c3d34a689c3cec947d649fc1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:41:50.699894 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:50.701376 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:50.714264 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00e4af-7a1a-40df-a00f-26313a5052a3 service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:50.762308 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94eb8ac7bd934c318f0e52833928c589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:50.762914 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:50.763197 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:50.763472 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:50.765288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:50.826509 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:50.827950 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:50.837862 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1444/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:41:51.016437 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:51.017973 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:51.032692 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43df541-a1b5-423f-a238-7eb68bc3845b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:51.037662 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:51.038523 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:51.038981 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:51.039391 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] PATH_INFO: `/v3/projects/1f420f7f5f5947088f23da2dab9f0fd7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:51.040594 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f420f7f5f5947088f23da2dab9f0fd7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:51.042670 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:51.092056 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f420f7f5f5947088f23da2dab9f0fd7', 'request_id': 'req-31633004-009f-42ad-b933-3a2822ac25f0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.097716 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f420f7f5f5947088f23da2dab9f0fd7', 'request_id': 'req-31633004-009f-42ad-b933-3a2822ac25f0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.105035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f420f7f5f5947088f23da2dab9f0fd7', 'request_id': 'req-31633004-009f-42ad-b933-3a2822ac25f0'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.116661 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] The token cache is being invalidate because project 1f420f7f5f5947088f23da2dab9f0fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:41:51.116943 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.117600 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.119106 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1448/2892] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:41:51 2022] DELETE /identity/v3/projects/1f420f7f5f5947088f23da2dab9f0fd7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:41:51.123143 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:51.133904 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8c2a-deaf-4f2f-8cac-454877bfdac4 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:51.176212 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:41:51.197020 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:51.197599 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:51.197880 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:51.198156 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] PATH_INFO: `/v3/users/ed49602d429344f48ccb41c11f575c4c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:51.199319 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed49602d429344f48ccb41c11f575c4c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:51.201266 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:51.247527 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed49602d429344f48ccb41c11f575c4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.248181 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed49602d429344f48ccb41c11f575c4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.255411 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed49602d429344f48ccb41c11f575c4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.262947 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed49602d429344f48ccb41c11f575c4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.274856 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed49602d429344f48ccb41c11f575c4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.280836 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1445/2893] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:41:51 2022] DELETE /identity/v3/users/ed49602d429344f48ccb41c11f575c4c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:41:51.459392 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:41:51.460890 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:41:51.471488 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:41:51.477607 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633004-009f-42ad-b933-3a2822ac25f0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:41:51.537678 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:41:51.538358 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:41:51.538583 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:41:51.538854 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] PATH_INFO: `/v3/projects/e6dc7bd3aaf94f6db124873ade24362e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:41:51.540039 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6dc7bd3aaf94f6db124873ade24362e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:41:51.541947 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:41:51.591035 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6dc7bd3aaf94f6db124873ade24362e', 'request_id': 'req-75ad69bc-da9c-4bbb-a468-39a031900627'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.598529 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6dc7bd3aaf94f6db124873ade24362e', 'request_id': 'req-75ad69bc-da9c-4bbb-a468-39a031900627'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.607683 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6dc7bd3aaf94f6db124873ade24362e', 'request_id': 'req-75ad69bc-da9c-4bbb-a468-39a031900627'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.619675 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] The token cache is being invalidate because project e6dc7bd3aaf94f6db124873ade24362e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:41:51.619996 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.620621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-75ad69bc-da9c-4bbb-a468-39a031900627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:41:51.622147 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1449/2894] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:41:51 2022] DELETE /identity/v3/projects/e6dc7bd3aaf94f6db124873ade24362e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:42:01.277374 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:01.616856 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:08.106895 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:08.108353 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:08.178090 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bdca-4a12-4ab9-861c-2858a86d518f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:08.238512 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2892e755ce4743adbd131ebda92931d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:08.239158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:08.239437 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:08.239705 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:08.241559 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:08.254524 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:08.256149 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:08.267341 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1446/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 15:42:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 15:42:09.036166 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:09.036621 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:09.036731 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:09.055653 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] MFA Rules not processed for user `6a40f282d2a94625ac21fb298a46b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 15:42:09.057418 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:09.110060 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1450/2896] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 15:42:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 15:42:09.150369 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:09.151793 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:09.221070 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291da0fe-d168-41c5-ae4d-6af871f563fc service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:09.225908 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeec63e358424f57ae02d5693b359805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a947acc76624771ba3560291d433410', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:09.226516 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:09.226796 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:09.227068 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:09.235660 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:09.237247 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:09.245167 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1447/2897] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 15:42:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 15:42:16.008471 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:16.009972 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:16.079463 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9970b-86fc-4af1-8bad-8b36087b8c83 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:16.084110 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:16.084767 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:16.085052 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:16.085331 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] PATH_INFO: `/v3/users/658f2b944c994780b615b5c0affd4600` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:16.086532 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=658f2b944c994780b615b5c0affd4600)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:16.089102 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:16.138236 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658f2b944c994780b615b5c0affd4600'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.144606 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658f2b944c994780b615b5c0affd4600'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.158632 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658f2b944c994780b615b5c0affd4600'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.166975 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658f2b944c994780b615b5c0affd4600'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.167642 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658f2b944c994780b615b5c0affd4600'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.173849 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1451/2898] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:42:16 2022] DELETE /identity/v3/users/658f2b944c994780b615b5c0affd4600 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:42:16.345593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:16.362050 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc87397-6022-45e9-b01f-6a6edd8eceb0 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:16.381176 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:16.420158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:16.420724 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:16.421005 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:16.421280 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] PATH_INFO: `/v3/projects/f40879b7ec11451a82501bf14592c04a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:16.422693 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40879b7ec11451a82501bf14592c04a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:16.424620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:16.472562 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40879b7ec11451a82501bf14592c04a', 'request_id': 'req-e5b9a315-a441-4574-98c4-d5228c293030'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.478701 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40879b7ec11451a82501bf14592c04a', 'request_id': 'req-e5b9a315-a441-4574-98c4-d5228c293030'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.485916 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40879b7ec11451a82501bf14592c04a', 'request_id': 'req-e5b9a315-a441-4574-98c4-d5228c293030'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.496620 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] The token cache is being invalidate because project f40879b7ec11451a82501bf14592c04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:42:16.496903 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.497520 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.499008 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1448/2899] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:42:16 2022] DELETE /identity/v3/projects/f40879b7ec11451a82501bf14592c04a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:42:16.503171 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:16.504679 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:16.518136 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9128-5e3f-47e8-86f5-4e076368eadb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:16.538020 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:16.581100 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:16.581749 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:16.582034 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:16.582313 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] PATH_INFO: `/v3/users/dff09ec31d78433387b2a2d831b890f3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:16.583486 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff09ec31d78433387b2a2d831b890f3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:16.585442 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:16.629787 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff09ec31d78433387b2a2d831b890f3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.637434 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff09ec31d78433387b2a2d831b890f3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.650122 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff09ec31d78433387b2a2d831b890f3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.657882 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff09ec31d78433387b2a2d831b890f3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.658792 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-c26ccd89-cb42-4b69-a919-aa26417c4c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff09ec31d78433387b2a2d831b890f3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.665273 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1452/2900] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 15:42:16 2022] DELETE /identity/v3/users/dff09ec31d78433387b2a2d831b890f3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:42:16.835585 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 15:42:16.837080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 15:42:16.851158 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9a315-a441-4574-98c4-d5228c293030 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 15:42:16.911386 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddce10ac514a48a58f8b731d31a20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1de4e10bc0744aea22a689aefa058d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 15:42:16.912069 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 15:42:16.912347 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 15:42:16.912622 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] PATH_INFO: `/v3/projects/1b6573e415c34be2a6e743bac33ccdf2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 15:42:16.913788 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6573e415c34be2a6e743bac33ccdf2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 15:42:16.916256 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 15:42:16.973080 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6573e415c34be2a6e743bac33ccdf2', 'request_id': 'req-6dc9496d-716e-440f-9088-e30de9cf8893'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.979928 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6573e415c34be2a6e743bac33ccdf2', 'request_id': 'req-6dc9496d-716e-440f-9088-e30de9cf8893'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:16.981593 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:16.989087 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6573e415c34be2a6e743bac33ccdf2', 'request_id': 'req-6dc9496d-716e-440f-9088-e30de9cf8893'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:17.000851 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] The token cache is being invalidate because project 1b6573e415c34be2a6e743bac33ccdf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 15:42:17.001155 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:17.001770 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6dc9496d-716e-440f-9088-e30de9cf8893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 15:42:17.003288 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1449/2901] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 15:42:16 2022] DELETE /identity/v3/projects/1b6573e415c34be2a6e743bac33ccdf2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 15:42:26.661387 dv-cinder-841315-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 15:42:26.997686 dv-cinder-841315-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}